Image output apparatus, image output method and image output program

ABSTRACT

There is provided an image output apparatus including an image-capturing section that captures an image of the observed person, an observer&#39;s point of sight measuring section that measures a point of sight of the observer when the image-capturing section captures the image of the observed person, an output section that outputs the image of the observed person which is captured by the image-capturing section, and a point of sight output control section that causes the output section to output information indicating the point of sight of the observer which is measured by the observer&#39;s point of sight measuring section, together with the image of the observed person which is captured by the image-capturing section.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese PatentApplications No. 2006-086703 filed on Mar. 27, 2006 and No. 2007-042174filed on Feb. 22, 2007, the contents of which are incorporated herein byreference.

BACKGROUND

1. Technical Field

The present invention relates to an image output apparatus, an imageoutput method and an image output program. More particularly, thepresent invention relates to an image output apparatus, an image outputmethod and an image output program for assisting an observer whoobserves an observed person.

2. Related Art

A medical system has been proposed which records a medical image inassociation with observed status information obtained when a medicaldoctor interprets the medical image (see Patent Document 1, forexample). According to the invention disclosed by Patent Document 1, theobserved status information recorded includes information regarding adevice used for displaying the medical image, the operation history ofthe displaying device, and point of sight information of the medicaldoctor. The operation history is used to reproduce how long the medicaldoctor observes the image. The point of sight information includesinformation that associates information regarding the line of sight ofthe medical doctor with a position on the display screen, a time periodfor which the medical doctor maintains observation with a particularpoint of sight, and the movement rate of the medical doctor's eyeball.The point of sight information is used to reproduce the position on themedical image the medical doctor pays attention to.

[Patent Document 1] Unexamined Japanese Patent Application PublicationNo. 2004-267273

In the course of diagnosing a patient who is an observed person, whatplays an important role in knowing the symptoms of the observed personis the movement of the observed person noticed by a medical doctor, whois an observer, while the medical doctor diagnoses the observed person,for example, the movement of the line of sight of the observed person.According to the invention disclosed in Patent Document 1, however, whatthe observer notices while interpreting the medical image is reproducedbased on the observed status information. Therefore, the invention cannot reproduce a moving part of the observed person which catches theobserver's attention while the observer diagnoses the observed person.There is another problem for the invention disclosed in PatentDocument 1. What plays an important role in knowing the symptoms of theobserved person also includes a change in the facial expression of theobserved person which is triggered by a question the observer asks tothe observed person while observing the observed person. The inventiondisclosed in Patent Document 1 can not reproduce the facial expressionworn by the observed person in response to the observer's question.

SUMMARY

In view of the above, an object of the present invention is to providean image output apparatus, an image output method and an image outputprogram which can solve the above-mentioned problems. This object isachieved by combining the features recited in the independent claims.The dependent claims define further effective specific example of thepresent invention.

To solve the above-mentioned problem, a first embodiment of the presentinvention provides an image output apparatus for assisting an observerwho observes an observed person. The image output apparatus includes animage-capturing section that captures an image of the observed person,an observer's point of sight measuring section that measures a point ofsight of the observer when the image-capturing section captures theimage of the observed person, an output section that outputs the imageof the observed person which is captured by the image-capturing section,and a point of sight output control section that causes the outputsection to output information indicating the point of sight of theobserver which is measured by the observer's point of sight measuringsection, together with the image of the observed person which iscaptured by the image-capturing section. Here, the point of sight outputcontrol section may cause the output section to output a mark indicatingthe point of sight of the observer which is measured by the observer'spoint of sight measuring section, in such a state that the mark isoverlapped onto the image of the observed person which is captured bythe image-capturing section. The image output apparatus may furtherinclude a point of sight image extracting section that extracts apartial image including the point of sight of the observer which ismeasured by the observer's point of sight measuring section, from theimage of the observed person which is captured by the image-capturingsection. Here, the point of sight output control section may cause theoutput section to output an image obtained by enlarging the partialimage extracted by the point of sight image extracting section, togetherwith the image of the observed person which is captured by theimage-capturing section.

The image output apparatus may further include a view measuring sectionthat measures a view of the observer. Here, the point of sight imageextracting section may extract an image corresponding to the view of theobserver which is measured by the view measuring section, from the imageof the observed person which is captured by the image-capturing section.The image output apparatus may further include an observed person'sinformation storing section that stores thereon the image of theobserved person which is captured by the image-capturing section inassociation with the observed person, and a previous image outputcontrol section that, when the image-capturing section captures a newimage of the observed person, causes the output section to output aprevious image of the observed person which is stored on the observedperson's information storing section in association with the observedperson, together with the new image of the observed person which iscaptured by the image-capturing section. The observed person'sinformation storing section may further store thereon the informationindicating the point of sight of the observer which is measured by theobserver's point of sight measuring section in association with theobserved person, and the previous image output control section may causethe output section to output the information indicating the point ofsight of the observer, together with the previous image of the observedperson.

The image output apparatus may further include a sound recording sectionthat records sound information of the observer when the image-capturingsection captures the image of the observed person, an image storingsection that stores thereon the sound information of the observer whichis recorded by the sound recording section, in association with theimage of the observed person which is captured by the image-capturingsection, a sound information comparing section that compares soundinformation of the observer which is newly recorded by the soundrecording section with the sound information of the observer which ispreviously recorded by the sound recording section and stored on theimage storing section, and a previous image extracting section thatextracts, based on a result of the comparison done by the soundinformation comparing section, a previous image of the observed personwhich is stored on the image storing section in association withprevious sound information of the observer which shows a highercoincidence level than a predetermined level with the sound informationof the observer which is newly recorded by the sound recording section.Here, the previous image output control section may cause the outputsection to output the previous image of the observed person which isextracted by the previous image extracting section, together with thenew image of the observed person which is captured by theimage-capturing section.

The output section may be a display section. The image output apparatusmay further include an observed person's information storing sectionthat stores the image of the observed person which is captured by theimage-capturing section in association with the point of sight of theobserver which is measured by the observer's point of sight measuringsection, a previous image output control section that reads the image ofthe observed person which is stored on the observed person's informationstoring section, and causes the display section to display the readimage of the observed person, and a viewer's point of sight measuringsection that measures a point of sight of a viewer who views the imageof the observed person which is displayed on the display section. Here,the observed person's information storing section may further storethereon information indicating the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section, inassociation with the image of the observed person which is read by theprevious image output control section.

The image output apparatus may further include a viewer's point of sightjudging section that judges whether the point of sight of the observerwhich is measured by the observer's point of sight measuring sectioncoincides with the point of sight of the viewer which is measured by theviewer's point of sight measuring section, and a viewed image extractingsection that, when the viewer's point of sight judging section judgesnegatively, extracts an image corresponding to information indicatingthat the points of sight do not coincide with each other. Here, theprevious image output control section may cause the output section tooutput the image which corresponds to the information indicating thatthe points of sight do not coincide with each other and is thusextracted by the viewed image extracting section.

The viewer's point of sight judging section may judge that the point ofsight of the observer which is measured by the observer's point of sightmeasuring section does not coincide with the point of sight of theviewer which is measured by the viewer's point of sight measuringsection, when the point of sight of the viewer is not present within apredetermined region including the point of sight of the observer. Whenthe point of sight of the observer which is measured by the observer'spoint of sight measuring section moves to a plurality of differentpositions, the viewer's point of sight judging section may judge thatthe point of sight of the observer coincides with the point of sight ofthe viewer which is measured by the viewer's point of sight measuringsection if the point of sight of the viewer moves to vicinities whichrespectively correspond to the plurality of different positions. Theviewer's point of sight judging section may judge that the point ofsight of the observer coincides with the point of sight of the viewer ifthe point of sight of the viewer which is measured by the viewer's pointof sight measuring section moves to the vicinities in a same order asthe point of sight of the observer.

The viewer's point of sight judging section may judge that the point ofsight of the observer coincides with the point of sight of the viewer ifthe point of sight of the observer which is measured by the observer'spoint of sight measuring section and the point of sight of the viewerwhich is measured by the viewer's point of sight measuring section areboth present in a predetermined region of the image of the observedperson which is captured by the image-capturing section. Thepredetermined region of the image may include at least one of an eye, amouth and a hand of the observed person.

The point of sight output control section may cause the output sectionto output at least one of (i) a mark indicating a position of the pointof sight of the observer which is measured by the observer's point ofsight measuring section and (ii) a mark indicating a position of thepoint of sight of the viewer which is measured by the viewer's point ofsight measuring section, in such a state that the at least one of themarks is overlapped onto the image of the observed person.

A second embodiment of the present invention provides an image outputmethod for assisting an observer who observes an observed person. Theimage output method includes capturing an image of the observed person,measuring a point of sight of the observer during the image capturing,and outputting information indicating the point of sight of the observerwhich is measured in the point of sight measuring, together with theimage of the observed person which is captured in the image capturing.

A third embodiment of the present invention provides an image outputprogram for an image output apparatus that assists an observer whoobserves an observed person. The image output program causes the imageoutput apparatus to function as an image-capturing section that capturesan image of the observed person, an observer's point of sight measuringsection that measures a point of sight of the observer when theimage-capturing section captures the image of the observed person, anoutput section that outputs the image of the observed person which iscaptured by the image-capturing section, and a point of sight outputcontrol section that causes the output section to output informationindicating the point of sight of the observer which is measured by theobserver's point of sight measuring section, together with the image ofthe observed person which is captured by the image-capturing section.

Here, all the necessary features of the present invention are not listedin the summary. The sub-combinations of the features may become theinvention.

According to the present invention, an image can be extracted whichshows an area of an observed person to which an observer pays attentionwhile observing the observed person, and displayed with the position ofthe observer's line of sight being overlapped thereon. Consequently, thepresent invention makes it possible to accurately know how the observerobserves the observed person.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view illustrating an image recording apparatus 10.

FIG. 2 is a block diagram illustrating the functional configuration ofthe image recording apparatus 10.

FIG. 3 is a block diagram illustrating the functional configuration ofan image-capturing unit 20.

FIG. 4 is a block diagram illustrating the functional configurations ofa storing unit 30 and a sound unit 40.

FIG. 5 illustrates the operation performed by a point of sight outputcontrol section 220.

FIG. 6 illustrates the operation performed by a previous image outputcontrol section 310.

FIG. 7 illustrates the structure of data stored on an image storingsection 320.

FIG. 8 is used to illustrate another example of the image-capturing unit20 relating to a first embodiment.

FIGS. 9A and 9B are used to illustrate the operation performed by aviewer's point of sight judging section 272.

FIG. 10 illustrates exemplary data stored on an observed person'sinformation storing section 300.

FIG. 11 is used to illustrate the operation in which an image isextracted by a viewed image extracting section 312 and the extractedimage is output by an output section 50.

FIG. 12 illustrates an exemplary functional configuration of theimage-capturing unit 20 relating to a second embodiment.

FIG. 13 is used to illustrate the operation performed by a point ofsight moving method learning section 216.

FIG. 14 is used to illustrate the operation performed by the point ofsight moving method learning section 216.

FIG. 15 is used to illustrate the operation performed by the point ofsight moving method learning section 216.

FIGS. 16A and 16B illustrate a case where the image recording apparatus10 is used for a video conference.

FIG. 17 illustrates exemplary functional configurations of theimage-capturing unit 20 relating to a third embodiment and an input unit60 relating to a second embodiment.

FIGS. 18A and 18B are used to illustrate the operation performed by anobserver's point of sight judging section 250.

FIG. 19 is used to illustrate the operation performed by the observer'spoint of sight judging section 250.

FIG. 20 illustrates one example of data stored on an image storingsection 320.

FIG. 21 illustrates exemplary functional configurations of theimage-capturing unit 20 relating to a fourth embodiment and input unit60 relating to a third embodiment in the image recording apparatus 10.

FIG. 22 illustrates a writing section 614.

FIG. 23 illustrates exemplary data stored on the image storing section320.

FIG. 24 is used to illustrate an electronic medical record 500.

FIGS. 25A and 25B are used to illustrate the electronic medical record500.

FIG. 26 illustrates an exemplary functional configuration of theimage-capturing unit 20 relating to a fifth embodiment.

FIG. 27 illustrates an exemplary functional configuration of the inputunit 60 relating to a fourth embodiment.

FIG. 28 illustrates an exemplary functional configuration of ananalyzing unit 70 relating to a second embodiment.

FIG. 29 is used to illustrate the function of a physical statuscalculating section 725.

FIG. 30 illustrates an exemplary structure of data stored on a physicalstatus data storing section 720.

FIG. 31 is a block diagram illustrating the hardware configuration ofthe image recording apparatus 10.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, some embodiments of the present invention will bedescribed. The embodiments do not limit the invention according to theclaims, and all the combinations of the features described in theembodiments are not necessarily essential to means provided by aspectsof the invention.

FIGS. 1 and 2 schematically illustrate an image recording apparatus 10relating to a first embodiment of the present invention. The imagerecording apparatus 10 includes therein an image-capturing unit 20, astoring unit 30, a sound unit 40, an output section 50, an input unit60, and an analyzing unit 70. The image-capturing unit 20 relating tothe first embodiment includes therein a plurality of image-capturingsections. For example, the image-capturing unit 20 includes animage-capturing section 202 for detecting the point of sight of anobserver 910, an image-capturing section 208 for measuring the areawithin which the observer 910 is present, an image-capturing section 204for detecting the point of sight of an observed person 915, and animage-capturing section 206 for measuring the area within which theobserved person 915 is present. The image-capturing sections 202 to 208may be each a three-dimensional camera, or a camera for capturingtwo-dimensional images. The output section 50 is a display, for example.The input unit 60 includes therein a digital input section 612 and awriting section 614. The digital input section 612 may include akeyboard and a mouse, and the writing section 614 may be a pen, forexample. The sound unit 40 may include therein a sound collectingsection 405. Note that the image recording apparatus 10 relating to thepresent embodiment is an example of an image output apparatus. The imagerecording apparatus 10 relating to the first embodiment aims to detectan area of the observed person 915 to which the observer 910 paysattention while the observer 910 is observing the observed person 915and to display an image showing the area of the observed person 915 towhich the observer 910 pays attention with the point of sight of theobserver 910 being overlapped onto the image. In addition, the firstembodiment aims to provide an image output apparatus which detects thepoint of sight of a viewer who views a recorded image of the observedperson 915 on the output section 50 so as to enable the viewer to lookat the point of sight of the observer 910.

The image-capturing unit 20 measures the points of sight of the observer910 and the observed person 915 by means of the corneal reflex method orby detecting a change in light intensity distribution of an image of theeyeball area. Also, the image-capturing unit 20 uses the image of theobserved person 915 captured by the image-capturing section 206 and theimage captured by the image-capturing section 202 for detecting thepoint of sight of the observer 910, in order to identify the area of theobserved person 915 which is observed by the observer 910.

Also, the image recording apparatus 10 uses the image captured by theimage-capturing section 208 for identifying the area, within theimage-capturing target area, in which the observer 910 is present and animage captured by the image-capturing section 204 for measuring thedirection of the line of sight of the observed person 915, in order toidentify an area of the observer 910 or an area of a space other thanthe observer 910 which the observed person 915 looks at. Here, theimage-capturing sections 202 to 208 of the image-capturing unit 20 mayinclude an omnidirectional camera. When any one of the image-capturingsections 202 to 208 is an omnidirectional camera, the singleomnidirectional camera can capture the images for measuring thedirections of the lines of sight of the observer 910 and observed person915 and the images for measuring the positions, within theimage-capturing target area, where the observer 910 and observed person915 are respectively present. Therefore, the image recording apparatus10 is not required to include therein a plurality of image-capturingsections such as the image-capturing section 202 as mentioned above.

When the observer 910 moves the line of sight to the area within whichthe observed person 915 is present, the image recording apparatus 10captures an image showing a predetermined area of the observed person915 which includes the point of sight of the observer 910. The imagerecording apparatus 10 then displays, on the output section 50, thecaptured image showing the observed person 915, with a mark indicatingthe point of sight of the observer 910. In this case, the imagerecording apparatus 10 may display, side by side, an image showing theobserved person 915 and an enlarged image obtained by enlarging at apredetermined enlarging ratio the captured partial image of the observedperson 915. According to the image recording apparatus 10, theimage-capturing section 206 captures the image of the observed person915 when the observer 910 writes the symptoms of the observed person 915into an electronic medical record by way of the digital input section612 such as a keyboard and a mouse, or when the observer 910 writes thesymptoms into a medical record using the writing section 614 such as apen. In addition, the image recording apparatus 10 obtains, through thesound collecting section 405, the sound uttered by the observer 910 toask a question to the observed person 915. In this way, the imagerecording apparatus 10 displays, side by side on the output section 50,an image of the observed person 915 which is captured when the observer910 previously asks the same question to the observed person 915 and thecurrent image of the observed person 915.

In the image recording apparatus 10, the analyzing unit 70 analyzes theimages captured by the image-capturing sections 204 and 206 to determinethe physical status of the observed person 915. For example, based onthe image of the observed person 915 captured by the image-capturingsection 204, the analyzing unit 70 obtains information about theobserved person 915 such as a change in the size of the pupils, themovement of the line of sight, the moving rate of the line of sight, andthe number of blinks, so as to identify the disease of the observedperson 915. When the image-capturing section 204 or 206 includes thereina temperature measuring section for measuring the temperature of theface of the observed person 915, the analyzing unit 70 may also use themeasured temperature of the face of the observed person 915 to identifythe disease of the observed person 915. The analyzing unit 70 displaysthe identified disease on the display screen. Specifically speaking, theanalyzing unit 70 displays, on the display screen, information includingthe name and symptoms of the disease. The analyzing unit 70 may displayon the display screen multiple disease candidates which the observedperson 915 may possibly suffer from.

According to the image recording apparatus 10 of the first embodiment,the point of sight of the observer 910 can be overlapped onto the imageof the observed person 915 when displayed. In this way, the imagerecording apparatus 10 makes it possible to easily know the area of theobserved person 915 on which the observer 910 focuses while observingthe observed person 915. For example, a moving image of the observedperson 915 can be captured and displayed on the output section 50 withthe movement of the point of sight of the observer 910 being overlappedonto the moving image. Therefore, the image recording apparatus 10 canreproduce how the observer 910 observes and diagnoses the observedperson 915. Also, the image recording apparatus 10 can capture an imageshowing how the observed person 915 behaves while the observer 910writes the result of the observation into the electronic medical record.In this way, even if the observer 910 misses a change in the facialexpressions of the observed person 915, the image recording apparatus 10makes it possible to easily retrieve and reproduce, after theobservation, an image showing the missed facial expressions of theobserved person 915. In addition, the image recording apparatus 10 candisplay side by side the previous and current images of the observedperson 915, and therefore makes it possible to accurately know a changein the symptoms of the observed person 915.

FIG. 3 illustrates an exemplary functional configuration of the firstembodiment of the image-capturing unit 20. The image-capturing unit 20includes therein the image-capturing sections 202, 204, 206 and 208, anobserver's point of sight measuring section 210, an observed person'sposition measuring section 212, an observed person's point of sightmeasuring section 214, a point of sight output control section 220, aview measuring section 230, a point of sight image extracting section240, and an observer's position measuring section 270. FIG. 4illustrates exemplary functional configurations of the storing unit 30and sound unit 40. The storing unit 30 includes therein an observedperson's information storing section 300, a previous image outputcontrol section 310, and an image storing section 320. The sound unit 40includes therein a sound recording section 400, a sound informationcomparing section 410, and a previous image extracting section 420.

The image-capturing section 204 captures an image of the observed person915. The image-capturing section 204 may capture a moving image of theobserved person 915. The image-capturing section 204 supplies thecaptured image of the observed person 915 to the observed person's pointof sight measuring section 214 and the image storing section 320 of thestoring unit 30. The image-capturing section 206 captures an image foridentifying a position at which the observed person 915 is presentwithin the predetermined image-capturing target area. Theimage-capturing section 206 supplies the captured image to the observedperson's position measuring section 212. The image-capturing section 202captures an image of the observer 910. The image-capturing section 202supplies the captured image to the observer's point of sight measuringsection 210 and the view measuring section 230. The image-capturingsection 208 captures an image for identifying the position at which theobserver 910 is present within the predetermined image-capturing targetarea. The image-capturing section 208 supplies the captured image to theobserver's position measuring section 270. Here, the image-capturingsections 202, 204, 206 and 208 may each capture a moving image of acorresponding one of the observer 910 and observed person 915.

When capturing moving images, the image-capturing sections 202, 204, 206and 208 each may supply to one or more corresponding constituents aplurality of images composing the corresponding moving image. Here, theimages composing the moving image may include any type of frame images,field images, and other moving-image composing images in a differentformat. Here, the image recording apparatus 10 may additionally includetherein a blink measuring section for measuring the number of blinks ofthe observed person 915. Such a blink measuring section measures thenumber of times the observed person 915 blinks within a predeterminedperiod of time. The blink measuring section supplies the measured numberof blinks of the observed person 915 to the observed person'sinformation storing section 300.

The observer's point of sight measuring section 210 measures the pointof sight of the observer 910 when the image-capturing sections 204 and206 capture the images of the observed person 915. Specificallyspeaking, the observer's point of sight measuring section 210 mayinclude therein a first observer's line of sight measuring section formeasuring the line of sight of the left eye of the observer 910 and asecond observer's line of sight measuring section for measuring the lineof sight of the right eye of the observer 910. The first and secondobserver's line of sight measuring sections use the image captured bythe image-capturing section 202 to measure the directions of the linesof sight of the left and right eyes of the observer 910. Here, theobserver's point of sight measuring section 210 may measure thedirections of the lines of sight of the observer 910 based on thecorneal reflex method. Alternatively, the first and second observer'sline of sight measuring sections extract the images showing the eyeballsfrom the images of the left and right eyes of the observer 910, anddetect changes in the light intensity distributions in the extractedimages showing the eyeballs. The first and second observer's line ofsight measuring sections may calculate the directions of the lines ofsight of the left and right eyes of the observer 910 based on thedetected light intensity distributions. For example, the firstobserver's line of sight measuring section calculates the lightintensity distribution of the image showing the eyeball, which varies inresponse to the position of the iris or pupil, in terms of the verticaland horizontal directions of the eyeball of the left eye.

To be specific, the directions of the lines of sight of the observer 910are each identified by comparing the light intensity distribution of theeyeball image which is observed when the line of sight of the observer910 extends in the front direction of the observer 910 and the lightintensity distribution of the eyeball image which is observed when theline of sight of the observer 910 extends in a direction other than thefront direction. For example, the direction in which the iris or pupilof the left eye faces, that is to say, the direction of the line ofsight of the left eye can be calculated by comparing the light intensitydistribution of the eyeball image of the left eye which is calculated bythe first observer's line of sight measuring section, with the lightintensity distribution of the left eyeball image which is observed whenthe line of sight of the left eye of the observer 910 extends in thefront direction of the observer 910. It should be noted that the firstobserver's line of sight measuring section may judge that the line ofsight of the left eye extends in the front direction of the observer 910when detecting that the light intensity distribution of the eyeballimage which is calculated in terms of the vertical and horizontaldirections of the eyeball of the left eye is substantially even withrespect to the central point of the left eye. In the same manner, thesecond observer's line of sight measuring section calculates thedirection of the line of sight of the right eye. Here, the first andsecond observer's line of sight measuring sections may identify thedirections of the lines of sight of the left and right eyes of theobserver 910 based on the combination of the corneal reflex method andthe detection of the changes in the light intensity distributions of theeyeball images. The observer's point of sight measuring section 210identifies the point of sight of the observer 910 based on the lines ofsight of the left and right eyes. For example, the observer's point ofsight measuring section 210 may identify, as the point of sight of theobserver 910, a point at which the lines of sight of the left and righteyes intersect each other or a point in the vicinity of the intersectingpoint. When the lines of sight of the left and right eyes do notintersect each other, the observer's point of sight measuring section210 may identify, as the point of sight of the observer 910, themidpoint of a perpendicular line which extends from one of the lines ofsight to the other or a point in the vicinity of the midpoint.Alternatively, the point of sight of the observer 910 can be calculatedbased on the lines of sight calculated by the observer's point of sightmeasuring section 210 and three-dimensional coordinate data whichindicates the area within which the observer 910 is present andcalculated by the observer's position measuring section 270, asdescribed later. The observer's point of sight measuring section 210supplies information indicating the identified point of sight of theobserver 910 to the point of sight output control section 220 and pointof sight image extracting section 240. Here, the observed person's pointof sight measuring section 214 measures the point of sight of theobserved person 915 in the same manner as the observer's point of sightmeasuring section 210. The observed person's point of sight measuringsection 214 supplies information indicating the point of sight of theobserved person 915 to the point of sight output control section 220 andpoint of sight image extracting section 240.

The observed person's position measuring section 212 measures theposition, within the predetermined image-capturing target area, at whichthe observed person 915 is present based on the image of the observedperson 915 captured by the image-capturing section 206. Specificallyspeaking, the observed person's position measuring section 212 obtainscoordinate data in which the position at which the observed person 915is present is determined in accordance with a predeterminedthree-dimensional coordinate system. For example, the positions andimage-capturing directions of the image-capturing sections 202, 204, 206and 208 and the positions at which the observer 910 and observed person915 are present are determined in advance in accordance with thepredetermined three-dimensional coordinate system. To be specific, anarea occupied by the observed person 915 within the image-capturingtarget area of the image-capturing section 206 can be identified bycausing the observed person 915 to be present at a predeterminedposition. For example, the area occupied by the observed person 915within the image-capturing target area of the image-capturing section206 can be identified by using coordinate data by causing the observedperson 915 to be seated at a predetermined position. In the same manner,the area occupied by the observer 910 within the image-capturing targetarea of the image-capturing section 208 can be identified by usingcoordinate data by causing the observer 910 to be seated at apredetermined position. When the observed person 915 moves within theimage-capturing target area of the image-capturing section 206, theobserved person's position measuring section 212 may measure the areaoccupied by the observed person 915 within the image-capturing targetarea of the image-capturing section 206 in compliance with the movementof the observed person 915.

The image-capturing section 206 may be a three-dimensional camera. Theobserved person's position measuring section 212 calculates the positionat which the observed person 915 is present in a three-dimensionalcoordinate system, based on the image captured by the image-capturingsection 206. For example, the image-capturing section 206 detects, foreach pixel, a distance from the image-capturing section 206 to the areawithin the image-capturing target area in which the observed person 915is present. Based on the distance to the observed person 915 which isdetected by the image-capturing section 206, the observed person'sposition measuring section 212 may calculate coordinate data indicatingthe position at which the observed person 915 is present within thethree-dimensional coordinate system. The observed person's positionmeasuring section 212 supplies the calculation result to the point ofsight output control section 220 and point of sight image extractingsection 240. Also, the observed person's position measuring section 212may supply the calculation result to the observer's point of sightmeasuring section 210. The observer's position measuring section 270measures the position, within the image-capturing target area of theimage-capturing section 208, at which the observer 910 is present. Theobserver's position measuring section 270 may calculate coordinate dataindicating the position within a three-dimensional coordinate system atwhich the observer 910 is present, in the same manner as the observedperson's position measuring section 212. The observer's positionmeasuring section 270 supplies the result of the calculation to thepoint of sight output control section 220 and point of sight imageextracting section 240. Since the image-capturing direction of theimage-capturing section 202 is determined in advance in accordance withthe three-dimensional coordinate system, the observer's point of sightmeasuring section 210 can calculate coordinate data indicating thedirection of the line of sight of the observer 910 based on the imagecaptured by the image-capturing section 202. In the same manner, theobserved person's point of sight measuring section 214 can calculatecoordinate data indicating the direction of the line of sight of theobserved person 915 based on the image captured by the image-capturingsection 204.

Here, the observer's point of sight measuring section 210 identifieswhich area of the observed person 915 has the point of sight of theobserver 910, based on the coordinate data which is calculated by theobserved person's position measuring section 212 and indicates the areain which the observed person 915 is present and the direction of theline of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210. Specifically speaking, theobserver's point of sight measuring section 210 may determine, as thepoint of sight of the observer 910, an intersection of the area of theobserved person 915 which is identified by the coordinate data which iscalculated by the observed person's position measuring section 212 andindicates the area in which the observed person 915 is present and aline extending from the eyes of the observer 910 along the direction ofthe line of sight of the observer 910 which is measured by theobserver's point of sight measuring section 210. The observer's point ofsight measuring section 210 obtains the coordinate data of theintersection as the coordinate data indicating the point of sight of theobserver 910. The observed person's point of sight measuring section 214may obtain the coordinate data indicating the point of sight of theobserved person 915 in the same manner as the observer's point of sightmeasuring section 210.

Also, the observer's point of sight measuring section 210 extractsinformation about the observer 910 such as the outline of the face, theshapes of the eyes, and the characteristic points of the eyes (the innerand outer corners of the eyes). When the observer 910 moves the head,the observer's point of sight measuring section 210 may trace theextracted characteristic points of the eyes and the like so as to followthe movement of the observer 910, and measure the line of sight of theobserver 910. For example, the observer's point of sight measuringsection 210 may include therein an observer's characteristic pointextracting section for extracting the characteristic points of the eyesof the observer 910 and the like and a movement control section formoving the image-capturing section 202 in accordance with the movementof the characteristic points of the eyes and the like of the observer910 which are extracted by the observer's characteristic pointextracting section. With this configuration, even when the observer 910moves the head, the observer's point of sight measuring section 210 canappropriately measure the point of sight of the observer 910 by movingthe image-capturing section 202 in accordance with the movement of thehead of the observer 910. The observer's point of sight measuringsection 210 supplies the measured point of sight of the observer 910 tothe point of sight output control section 220 and point of sight imageextracting section 240.

Here, the observer 910 may remotely observe the observed person 915. Forexample, an image recording apparatus 10 is provided for the observer910 and another image recording apparatus 10 is provided for theobserved person 915, and the image recording apparatuses 10 areconnected to each other by a network such as the Internet. If this isthe case, the image-capturing sections 204 and 206 for capturing theimages of the observed person 915 supply, via the network, the capturedimages of the observed person 915 to the output section 50 of the imagerecording apparatus 10 for the observer 910. Thus, the observer 910observes the images of the observed person 915 displayed on the outputsection 50. In this case, the observer's point of sight measuringsection 210 measures which area of the observed person 915 displayed onthe output section 50 is focused by the observer 910.

The view measuring section 230 identifies the view of the observer 910.To be specific, the view measuring section 230 analyzes the imagecaptured by the image-capturing section 202, so as to identify an angleof convergence based on the directions of the lines of sight of the leftand right eyes of the observer 910. For example, the view measuringsection 230 may calculate the angle of convergence formed by thedirections of the lines of sight of the left and right eyes of theobserver 910 based on the image of the observer 910 captured by theimage-capturing section 202. Here, the view measuring section 230 mayreceive the information indicating the directions of the lines of sightof the observer 910 which are measured by the observer's point of sightmeasuring section 210, and calculate the angle of convergence based onthe received information indicating the directions of the lines ofsight. Alternatively, the view measuring section 230 may measure thefocal distance of the left and right eyes of the observer 910, andidentify, as the view of the observer 910, the area within which theeyes of the observer 910 focus together. As another example, the viewmeasuring section 230 measures a still time period during which the lineof sight of the observer 910 stays still. Specifically speaking, theview measuring section 230 may measure the still time period of the lineof sight of the observer 910, and identify, as the view of the observer910, an area for which the measured still time period is longer than apredetermined time period. Here, the predetermined time period may beone second, for example. The view measuring section 230 supplies theidentified view of the observer 910 to the point of sight imageextracting section 240.

The point of sight image extracting section 240 extracts a partialimage, from the image of the observer person 915 which is received fromthe image-capturing section 204, which includes the point of sight ofthe observer 910 which is measured by the observer's point of sightmeasuring section 210. Also, the point of sight image extracting section240 may extract an image corresponding to the view of the observer 910which is measured by the view measuring section 230, from the image ofthe observed person 915 which is received from the image-capturingsection 204. The point of sight image extracting section 240 suppliesthe extracted image to the point of sight output control section 220.The point of sight output control section 220 causes the output section50 to output the information indicating the point of sight of theobserver 910 which is received from the observer's point of sightmeasuring section 210, together with the image of the observed person915 which is captured by the image-capturing section 204.

Specifically speaking, the point of sight output control section 220causes the output section 50 to output a mark indicating the point ofsight of the observer 910 which is received from the observer's point ofsight measuring section 210, in such a state that the mark is overlappedonto the image of the observed person 915 which is captured by theimage-capturing section 204. The point of sight output control section220 may cause the output section 50 to output an enlarged image obtainedby enlarging the image extracted by the point of sight image extractingsection 240 at a predetermined enlarging ratio, together with the imageof the observed person 915 which is captured by the image-capturingsection 204. Here, the input unit 60 may include therein an enlargingratio input section for allowing the observer 910 to input an enlargingratio used by the point of sight output control section 220 to enlargethe image extracted by the point of sight image extracting section 240.Since the observer 910 is allowed to input a desired enlarging ratiointo the enlarging ratio input section, the point of sight outputcontrol section 220 can cause the output section 50 to output the imageof the observed person 915 which has been enlarged to a desired size.

The observed person's information storing section 300 stores thereon theimage of the observed person 915 which is received from theimage-capturing section 204, in association with the observed person915. Also, the observed person's information storing section 300 storesthereon the information indicating the point of sight of the observer910 which is received from the observer's point of sight measuringsection 210, in association with the observed person 915. Furthermore,the observed person's information storing section 300 may store thereonthe number of blinks of the observed person 915 which is received fromthe blink measuring section, in association with the time at which theobserved person 915 is observed. The observed person's informationstoring section 300 supplies the image of the observed person 915 andinformation indicating the number of blinks of the observed person 915to the previous image output control section 310, under the control ofthe previous image output control section 310. The sound recordingsection 400 records sound information relating to the observer 910 whenthe image-capturing section 204 captures the image of the observedperson 915. The sound recording section 400 records sound informationobtained when the observer 910 asks a question to the observed person915, for example. The sound recording section 400 supplies the recordedsound information to the image storing section 320 and sound informationcomparing section 410.

The image storing section 320 stores thereon the image of the observedperson 915 which is received from the image-capturing section 204 andthe sound information which is received from the sound recording section400, in association with each other. The image storing section 320 maystore thereon the image and sound information, in association with thetime at which the sound recording section 400 records the soundinformation. The image storing section 320 supplies the soundinformation to the sound information comparing section 410, under thecontrol of the sound information comparing section 410. When theimage-capturing section 204 captures a new image of the observed person915, the previous image output control section 310 causes the outputsection 50 to output a previous image of the observed person 915 beingstored on the observed person's information storing section 300 inassociation with the observed person 915, together with the image of theobserved person 915 which is newly captured by the image-capturingsection 204 while the observer 910 observes the observed person 915.

Here, the previous image of the observed person 915 which is output bythe output section 50 under the control of the previous image outputcontrol section 310 may be an image of the observed person 915 which iscaptured when the observer 910 observes the observed person 915 lasttime, for example. Alternatively, the previous image may be an image ofthe observed person 915 which is captured at the first visit. Also, theprevious image output control section 310 may cause the output section50 to output information indicating the point of sight of the observer910, together with the previous image of the observed person 915. Theprevious image output control section 310 may cause the number of blinksof the observed person 915 to be displayed in the state of beingoverlapped on the image of the observed person 915. When theimage-capturing section 204 captures a new image of the observed person915, the previous image output control section 310 may cause the outputsection 50 to output the previous image of the observed person 915stored on the observed person's information storing section 300 inassociation with the observed person 915, the image of the observedperson 915 which is newly captured by the image-capturing section 204,and the difference between the previous and current numbers of blinks.

The sound information comparing section 410 compares the soundinformation of the observer 910 which is newly recorded by the soundrecording section 400 with the sound information of the observer 910which is previously recorded by the sound recording section 400 andstored on the image storing section 320. For example, the soundinformation comparing section 410 may include therein a wordpronunciation information storing section for storing, in associationwith words that are included in sound information, informationindicating frequency components observed when the words are pronouncedby a human. The sound information comparing section 410 analyzes thesound information which is newly received from the sound recordingsection 400 in terms of frequency components at predetermined timeintervals, to generate frequency component information. Subsequently,the sound information comparing section 410 compares the generatedfrequency component information with the information indicating thefrequency components of the words which is stored on the wordpronunciation information storing section. In this way, the soundinformation comparing section 410 calculates a coincidence level betweenthe sound information of the observer 910 which is newly recorded by thesound recording section 400 and the previous sound information of theobserver 910. The sound information comparing section 410 supplies thecalculated coincidence level to the previous image extracting section420.

When the coincidence level received from the sound information comparingsection 410 is equal to or higher than a predetermined level, theprevious image extracting section 420 extracts a previous image of theobserved person 915 which is stored on the image storing section 320 inassociation with the previous sound information of the observer 910which shows the received coincidence level. In this case, the previousimage output control section 310 may cause the output section 50 tooutput the previous image of the observed person 915 which is extractedby the previous image extracting section 420, together with the image ofthe observed person 915 which is newly captured by the image-capturingsection 204. The output section 50 outputs the image of the observedperson 915 which is received from the image-capturing section 204. Also,the output section 50 outputs the images received from the point ofsight output control section 220 and previous image extracting section420. In addition, the output section 50 may display the number of blinksof the observed person 915 which is measured by the blink measuringsection in such a state that the number of blinks is overlapped onto theimage of the observed person 915. Here, the output section 50 is amonitor or the like, for example.

In the case of the second or subsequent observation for the observedperson 915, the image recording apparatus 10 may ask questions to theobserved person 915 in place of the observer 910, based on the resultsof the previous observations which are done by the observer 910 on theobserved person 915. For example, the image recording apparatus 10 mayinclude therein a question storing section for storing thereonpredetermined questions in association with the behaviors of theobserved person 915. The image recording apparatus 10 then may displayan appropriate question selected from the questions stored on thequestion storing section in accordance with the symptoms of the observedperson 915, on the display screen which is shown as one example of theoutput section 50, so as to encourage the observed person 915 to answerto the displayed question and record the answer thereon. In this way,the observer 910 is not required to do a routine health interview. As aresult, the observer 910 can spend a longer time period on theobservation of the observed person 915 than otherwise.

The image recording apparatus 10 relating to the present embodimentjudges whether the observer 910 observes the entire body of the observedperson 915 or pays attention to a particular area of the observed person915, and obtains the point of sight of the observer 910 which isidentified when the observer 910 pays attention to the particular areaof the observed person 915. Thus, the image recording apparatus 10 canextract and display an image showing an area of the observed person 915which corresponds to the point of sight of the observer 910. With thisconfiguration, the image recording apparatus 10 enables the observer 910and other users to easily know which area of the observer person 915 theobserver 910 pays attention to during the observation. Also, the imagerecording apparatus 10 can display the point of sight of the observer910 together with the image of the observed person 915 which is capturedwhile the observer 910 observes the observed person 915. Therefore, theimage recording apparatus 10 makes it possible to reproduce how theobserver 910 observes the observed person 915.

Furthermore, the image recording apparatus 10 relating to the presentembodiment can identify the point of sight and view of the observer 910.When the point of sight of the observer 910 is present within the areaoccupied by the observed person 915, or when the observed person 915 ispresent within the view of the observer 910, the image recordingapparatus 10 can extract the image of the observed person 915. In thismanner, the image recording apparatus 10 can appropriately judge thatthe observer 910 pays attention to the observed person 915, andcorrectly extract the image of the observed person 915 which is capturedwhen the observer 910 observes the observed person 915. Consequently,the image recording apparatus 10 can extract an image useful for theobservation of the observed person 915.

In addition, the image recording apparatus 10 relating to the presentembodiment can display, side by side, the current image of the observedperson 915 and the previously captured image of the observed person 915.Therefore, the image recording apparatus 10 makes it possible to easilyknow the change of the status of the observed person 915. Also, theimage recording apparatus 10 relating to the present embodiment candisplay, side by side, the current image of the observed person 915which is captured when the observer 910 asks a certain question to theobserved person 915 and the image of the observed person 915 which iscaptured when the observer 910 previously asks the same question to theobserved person 915. In this way, the observer 910 can observe thestatus of the observed person 915 while comparing the current andprevious facial expressions of the observed person 915 which areobserved when the observer 910 asks the same question to the observedperson 915.

FIG. 5 illustrates an exemplary operation performed by the point ofsight output control section 220 relating to the present embodiment. Thepoint of sight output control section 220 causes the image-capturingsection 204 to capture an image of a person 900, who is the observedperson 915, when the point of sight of the observer 910 is presentwithin the area occupied by the person 900. Note that the point of sightoutput control section 220 judges whether the still time period of thepoint of sight which is measured by the view measuring section 230 isequal to or longer than a predetermined time period. When judgingpositively, the point of sight output control section 220 determinesthat the observer 910 pays attention to the area including the point ofsight. Subsequently, the point of sight output control section 220extracts the image of the person 900 and a partial image of the person900 which includes the point of sight.

A case is assumed where the point of sight of the observer 910 is atpositions indicated by a mark 800, a mark 805 and a mark 810, forexample. Here, if the still time period of the point of sight at theposition indicated by the mark 800 is equal to or longer than apredetermined time period, the point of sight output control section 220extracts an image of an area corresponding to an eye 950 of the person900 which includes the point of sight indicated by the mark 800.Following this, the point of sight output control section 220 enlargesthe extracted image to obtain an enlarged image 514, and outputs theenlarged image 514 to the output section 50. Here, the point of sightoutput control section 220 may cause the output section 50 to output animage 512 including the entire body of the person 900 along with theenlarged image 514. In the enlarged image 514, the region including theeye 950 of the person 900 is displayed in the enlarged state.

The point of sight output control section 220 relating to the presentembodiment can enlarge a portion of the area occupied by the observedperson 915 to which the observer 910 pays attention, and display, sideby side, the enlarged image and the image of the observed person 915.Thus, it is possible to reproduce the observation done by the observer910 in such a manner as to make it possible for a user of the imagerecording apparatus 10 to meticulously observe the area to which theobserver 910 pays attention while observing the observed person 915.Also, the user of the image recording apparatus 10 can learn theprocedure used by the observer 910 to do observation with reference tothe observation done by the observer 910 which is reproduced by theimage recording apparatus 10.

FIG. 6 illustrates an exemplary operation performed by the previousimage output control section 310 relating to the present embodiment. Theprevious image output control section 310 extracts a previous image of aperson 902, who is the observed person 915 being observed by theobserver 910, which is stored on the observed person's informationstoring section 300 in association with the person 902. Subsequently,the previous image output control section 310 causes the output section50 to output, side by side, a current image 520 of the person 902 whichis captured by the image-capturing section 204 and a previous image 522including the person 902. The previous image of the person 902 which isextracted by the previous image output control section 310, for example,the image 522, may be an image of the person 902 which is captured atthe first visit, an image of the person 902 which is captured when theperson 902 is observed last time, or an image of the person 902 which iscaptured at a date and a time designated by the observer 910.

To be specific, the input unit 60 provided in the image recordingapparatus 10 may include therein an image-capturing time input section.The image-capturing time input section allows the observer 910 to inputa date and a time at which an image of the observed person 915 iscaptured. The image-capturing time input section supplies the date andtime input by the observer 910 to the previous image output controlsection 310. The previous image output control section 310 extracts animage of the observed person 915 which is captured at the date and timereceived from the image-capturing time input section, and causes theoutput section 50 to output the extracted image.

The previous image output control section 310 relating to the presentembodiment can cause the output section 50 to display, side by side, thecurrent and previous images of the observed person 915. Therefore, theobserver 910 can perform accurate observation by comparing the currentand previous facial expressions of the person 902.

FIG. 7 illustrates an exemplary structure of data stored on the imagestoring section 320 relating to the present embodiment. The imagestoring section 320 stores thereon the sound information of the observer910 and the image of the observed person 915. To be specific, the imagestoring section 320 stores thereon, in association with a time at whichsound information 560 of the observer 910 is recorded by the soundrecording section 400, an image 522 of the observed person 915 which iscaptured by the image-capturing section 204 at the time and the soundinformation 560 of the observer 910.

For example, the sound recording section 400 records the soundinformation 560 which is obtained when the observer 910 asks a questionto the observed person 915. The sound recording section 400 supplies thesound information 560 to the image storing section 320 and soundinformation comparing section 410. The image storing section 320 storesthe image 522 of the observed person 915 which is received from theimage-capturing section 204 and the sound information 560 which isreceived from the sound recording section 400, in association with thetime at which the sound recording section 400 records the soundinformation 560. The image storing section 320 supplies the soundinformation 560 to the sound information comparing section 410 under thecontrol of the sound information comparing section 410. The soundinformation comparing section 410 compares sound information of theobserver 910 which is newly recorded by the sound recording section 400with the previous sound information 560 of the observer 910 which isstored on the image storing section 320, to calculate a coincidencelevel between the newly recorded sound information and the previoussound information 560. When the coincidence level calculated by thesound information comparing section 410 is equal to or higher than thepredetermined coincidence level, the previous image output controlsection 310 extracts the image 522 from the image storing section 320,and causes the output section 50 to output the extracted image 522together with the current image of the observed person 915. In this way,the image recording apparatus 10 makes it possible to later reproduce apreviously asked question and to reproduce the behavior of the observedperson 915 which is observed when the question is previously asked.

The image recording apparatus 10 relating to the present embodiment canbe utilized in the medical field. To be specific, the image recordingapparatus 10 is used as a diagnosis assisting apparatus that recordsthereon the contents of the diagnoses made for patients. For example,the image recording apparatus 10 is a diagnosis assisting apparatus thatrecords thereon the contents of the diagnoses made for patients, theobserved person is a patient suffering from a mental disease, and theobserver is a medical doctor who observes the patient. When the observedperson 915 is a patient suffering from a mental disease, the medicaldoctor often diagnoses the patient with focusing on an area of thepatient which is likely to show the tension. The image recordingapparatus 10 relating to the present embodiment makes it possible toeasily reproduce at a later time than the diagnosis which area of thepatient the medical doctor pays attention to during the diagnosis. Forthis reason, it is highly useful to use the image recording apparatus 10for assisting the diagnoses of patients suffering from mental diseases.

FIG. 8 is used to illustrate another example of the image-capturing unit20 relating to the first embodiment. The image-capturing unit 20 furtherincludes therein an image-capturing section 205, a viewer's point ofsight measuring section 271, and a viewer's point of sight judgingsection 272. Here, the storing unit 30 may further include therein aviewed image extracting section 312. As described above, the observedperson's information storing section 300 stores thereon the images ofthe observed person 915 which are captured by the image-capturingsections 204 and 206, in association with the information indicating thepoint of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210. Here, the previous image outputcontrol section 310 reads the image of the observed person 915 which isstored on the observed person's information storing section 300, andcauses a display section of the output section 50 to display the readimage. This operation is shown in FIG. 8. In this case, the point ofsight output control section 220 does not cause the output section 50 todisplay the mark indicating the position of the point of sight of theobserver 910. The image-capturing section 205 captures an image of theviewer who views the image of the observed person 915 which is displayedon the display section of the output section 50, and supplies thecaptured image of the viewer to the viewer's point of sight measuringsection 271.

The viewer's point of sight measuring section 271 measures the point ofsight of the viewer who views the image of the observed person 915 whichis displayed on the output section 50. Here, the viewer may or may notbe the observer 910. For example, when the observer 910 is a medicaldoctor, the viewer is another medical doctor, a nurse, or an intern. Theviewer's point of sight measuring section 271 identifies an area of theobserved person 915 displayed on the display section of the outputsection 50 in which the point of sight of the viewer is present, andsupplies information indicating the point of sight of the viewer to theviewer's point of sight judging section 272 and observed person'sinformation storing section 300. Here, the viewer's point of sightmeasuring section 271 may measure the point of sight of the viewer inthe same manner as the observer's point of sight measuring section 210,observed person's point of sight measuring section 214 and viewmeasuring section 230. In FIG. 8, the image-capturing section 205 isshown as a camera that is fixed at a remote position from the viewer,for example. However, the image-capturing section 205 may be ahead-mounted camera to be mounted on the head of the viewer. If such isthe case, the positions of the eyeballs of the viewer with respect tothe coordinate system of the camera are in advance calibrated. Also, thepositional relation between the viewer and the display screen of theoutput section 50 is identified in advance. Here, the point of sight ofthe viewer may be identified by measuring the movement of the eyeballsof the viewer based on, for example, the corneal reflex method. Theobserved person's information storing section 300 further stores thereonthe point of sight of the viewer which is measured by the viewer's pointof sight measuring section 271, in association with the image of theobserved person 915 which is read by the previous image output controlsection 310.

The viewer's point of sight judging section 272 judges whether the pointof sight of the observer 910 which is measured by the observer's pointof sight measuring section 210 and the point of sight of the viewerwhich is measured by the viewer's point of sight measuring section 271,and supplies the result of the judgment to the viewed image extractingsection 312. To be specific, the viewer's point of sight judging section272 judges whether the information indicating the point of sight of theobserver 910 which is stored on the observed person's informationstoring section 300 coincides with the point of sight of the viewer.When the viewer's point of sight judging section 272 judges negatively,the viewed image extracting section 312 extracts the image of theobserved person 915 which corresponds to the information indicating thepoints of sight do not coincide with each other. The previous imageoutput control section 310 causes the output section 50 to output theimage of the observed person 915 which corresponds to the informationindicating that the points of sight do not coincide with each other andthus is extracted by the viewed image extracting section 312. Thisconfiguration enables the viewer to look at the movement of the point ofsight of the observer 910 and the movement of the viewer's own point ofsight. According to the present embodiment, the image is output when thepoints of sight do not coincide with each other. In this way, the viewercan easily retrieve an image corresponding to a case where the vieweruses a different observation procedure from the observer 910. Thisconfiguration can help the viewer efficiently learn where to payattention to, especially when the viewer is an intern.

The point of sight output control section 220 may cause the outputsection 50 to output at least one of the marks respectively indicatingthe position of the point of sight of the observer 910 which is measuredby the observer's point of sight measuring section 210 and the positionof the point of sight of the viewer which is measured by the viewer'spoint of sight measuring section 271, in such a state that the mark isoverlapped onto the image of the observed person 915. Since the markindicating the point of sight of the observer 910 is displayed, theviewer can easily know which part the viewer is supposed to payattention to. In addition to the mark indicating the point of sight ofthe observer 910, the mark indicating the viewer's point of sight isalso displayed. In this way, the viewer can objectively realize anerror.

FIG. 9 is used to illustrate the operation performed by the viewer'spoint of sight judging section 272. The viewer's point of sight judgingsection 272 judges that the points of sight of the viewer 917 andobserver 910 do not coincide with each other, when the point of sight ofthe viewer which is measured by the viewer's point of sight measuringsection 271 is not present within a predetermined region including thepoint of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210. Here, the predetermined regionincluding the point of sight of the observer 910 which is measured bythe observer's point of sight measuring section 210 is determined basedon the position of the point of sight of the observer 910. For example,a region 521 is determined based on a point of sight 820 of the observerwhich is measured by the observer's point of sight measuring section 210as shown in FIG. 9A. When the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section 271 is notpresent within the region 521, the viewer's point of sight judgingsection 272 judges that the points of sight of the viewer 917 andobserver 910 do not coincide with each other.

A case is assumed where the point of sight of the observer 910 which ismeasured by the observer's point of sight measuring section 210 moves tomultiple different positions. In this case, the viewer's point of sightjudging section 272 may judge that the points of sight coincide witheach other if the point of sight of the viewer which is measured by theviewer's point of sight measuring section 271 moves to the vicinitieswhich respectively correspond to the positions of the point of sight ofthe observer 910. Here, the vicinities of the positions may be regionswhich are determined in accordance with the point of sight of theobserver 910 which is measured by the observer's point of sightmeasuring section 210. Furthermore, the viewer's point of sight judgingsection 272 may judge that the points of sight coincide with each otherwhen the point of sight of the viewer which is measured by the viewer'spoint of sight measuring section 271 moves to the vicinities in the sameorder as the point of sight of the observer 910. In the presentembodiment, the viewer's point of sight judging section 272 judgesnegatively since none of the reference numerals 826, 830 and 832indicating the point of sight of the viewer 917 is present within theregions 521, 523 and 524 determined based on the point of sight of theobserver 910.

The viewer's point of sight judging section 272 may judge that thepoints of sight coincide with each other when the point of sight of theviewer 917 moves to the vicinities of the respective positions of thepoint of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210, irrespective of the movement orderof the point of sight of the viewer 917. Here, the predetermined regionseach including the point of sight of the observer 910 may be defined bycircles, squares or rectangles with the point of sight of the observer910 being set as the center. In addition, the predetermined regions eachincluding the point of sight of the observer 910 may each have a sizedetermined in accordance with at least one of the movement rates of thepoints of sight of the observer 910 and viewer 917. For example, whenthe movement rate of the point of sight of the observer 910 is higherthan a predetermined rate, the viewer's point of sight judging section272 may set a larger size to the predetermined regions each includingthe point of sight of the observer 910. Meanwhile, when the movementrate of the point of sight of the viewer 917 is lower than apredetermined rate, the viewer's point of sight judging section 272 mayset a larger size to the predetermined regions each including the pointof sight of the observer 910. Here, the predetermined rate may be equalto the average rate of the movement of the point of sight of theobserver 910 displayed on the output section 50, or the average of themovement rates of the points of sight of multiple observers which arestored on the image storing section 320. Alternatively, thepredetermined rate may be equal to the average rate of the movement ofthe point of sight of the viewer 917, or the average of the movementrates of the points of sight of multiple viewers.

As an alternative example, the viewer's point of sight judging section272 may judge that the points of sight coincide with each other when thepoint of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210 and the point of sight of theviewer which is measured by the viewer's point of sight measuringsection 271 are both present within a predetermined region of the imageof the observed person 915 which is captured by the image-capturingsection 206, as described with reference to FIG. 9B. According to thepresent embodiment, the predetermined region of the image of theobserved person 915 is made up by a plurality of separate regions whichrespectively include therein at least the eyes, mouth and hands of theobserved person 915. In detail, the region including the eyes may be aregion corresponding to the black part (pupil and iris) of one of theeyes, a region corresponding to the black and white areas of one of theeyes, or a predetermined circular, rectangular or oval region includingthe black and white parts of one of the eyes. Alternatively, the regionincluding the eyes may be a predetermined circular, rectangular or ovalregion including both eyes. The region including the mouth may be aregion corresponding to the lips, a region defined by the lips, or apredetermined circular, rectangular or oval region including the lips.The region including the hands may be a region corresponding to the palmor back of one of the hands or a predetermined circular, rectangular oroval region including the region corresponding to the palm or back ofone of the hands. Alternatively, the region including the hands may be apredetermined circular, rectangular or oval region including the palmsor backs of both hands. The regions 526, 528 and 530 are shown in FIG.9B as examples of the regions respectively including the eyes, mouth,and hands of the observed person 915. According to the presentembodiment, since the points of sight of the observer 910 and viewer 917are both present in the region 526, the viewer's point of sight judgingsection 272 judges that the points of sight coincide with each other.With the above-described configuration, the image recording apparatus 10can easily make a judgment whether the points of sight coincide witheach other.

Alternatively, the viewer's point of sight judging section 272 may makethe judgment whether the points of sight coincide with each other byadditionally taking into consideration the order in which the point ofsight of the observer 910 moves. As another alternative example, theviewer's point of sight judging section 272 may make the judgmentwhether the points of sight coincide with each other based on therespective likelihoods at which the points of sight of the observer 910and viewer which are measured by the observer's point of sight measuringsection 210 and viewer's point of sight measuring section 271 arepresent within the predetermined region of the image of the observedperson 915 during a predetermined time period. Here, when the points ofsight move to multiple different positions during the predetermined timeperiod, the viewer's point of sight judging section 272 may calculatethose likelihoods by calculating a total time period for which each ofthe points of sight stays still in each of the multiple predeterminedregions, for example. The viewer's point of sight judging section 272obtains a coincidence level between the likelihoods which are calculatedfor the observer and viewer with respect to each predetermined region,and judges that the points of sight coincide with each other when thecoincidence level is equal to or higher than a predetermined level.

FIG. 10 illustrates an example of the data stored on the observedperson's information storing section 300. The observed person'sinformation storing section 300 relating to the present embodimentstores thereon, for each observed person, the image of the observedperson 915 captured by the image-capturing section 206 together with theinformation indicating the point of sight of the observer 910 which ismeasured by the observer's point of sight measuring section 210. Theobserved person's information storing section 300 relating to thepresent embodiment stores thereon the image of the observed person 915which is captured by the image-capturing section 206 together with theinformation indicating the time and the coordinate data indicating thepoint of sight of the observer 910. Also, the observed person'sinformation storing section 300 relating to the present embodimentstores thereon the coordinate data indicating the point of sight of theviewer 917 together with the coordinate data indicating the point ofsight of the observer 910.

FIG. 11 is used to illustrate the operation in which the image extractedby the viewed image extracting section 312 is output by the outputsection 50. Based on the judgment made by the viewer's point of sightjudging section 272, the viewed image extracting section 312 extracts animage of the observed person 915 which is stored in association with thecoordinate data indicating the point of sight that does not show thecoincidence, and the previous image output control section 310 causesthe output section 50 to output the image extracted by the viewed imageextracting section 312. Here, the previous image output control section310 may cause the output section 50 to output a plurality of imagesextracted by the viewed image extracting section 312 to reproduce asingle continuous moving image. Alternatively, the previous image outputcontrol section 310 may cause the output section 50 to play back theimages corresponding to no coincidence at a normal speed, and play backthe images corresponding to coincidence at a higher speed than normal.In this way, the image recording apparatus 10 relating to the presentembodiment can select and output the images corresponding to nocoincidence between the points of sight of the observer and viewerwithin a brief time period. This configuration enables the viewer toeasily look at the images corresponding to no coincidence.

FIG. 12 illustrates an exemplary functional configuration of theimage-capturing unit 20 relating to a second embodiment. Theimage-capturing unit 20 relating to the present embodiment includestherein an image-capturing section 200, the observer's point of sightmeasuring section 210, the observed person's position measuring section212, the observed person's point of sight measuring section 214, a pointof sight moving method learning section 216, the observer's point ofsight judging section 250, a point of sight moving method storingsection 255, and an image storing control section 260. The presentembodiment provides an image recording apparatus for accuratelyrecording the change in the facial expressions of the observed person915 during conversation. The image-capturing unit 20 relating to thepresent embodiment may further include some or all of the functions andconstituents of the units which together constitute the image recordingapparatus 10 described with reference to FIGS. 1 to 11. In the followingdescription, the constituents assigned with the same reference numeralsas in FIGS. 1 to 11 have substantially the same functions as thecorresponding constituents described with reference to FIGS. 1 to 11,and therefore not explained in detail.

When the image-capturing section 200 is an omnidirectional camera, forexample, the image-capturing section 200 may have the functions of theimage-capturing sections 202, 204, 206 and 208 described with referenceto FIGS. 1 to 11. Alternatively, the image-capturing unit 20 may includetherein, in place of the image-capturing section 200, theimage-capturing section 202 for detecting the point of sight of theobserver 910, the image-capturing section 208 for measuring the area inwhich the observer 910 is present, the image-capturing section 204 fordetecting the point of sight of the observed person 915, and theimage-capturing section 206 for measuring the area in which the observedperson 915 is present, similarly to the embodiment described withreference to FIGS. 1 to 11.

The image-capturing section 200 supplies the captured image of theobserved person 915 to the image storing control section 260 under thecontrol of the image storing control section 260. Also, theimage-capturing section 200 supplies the image used for measuring thepoint of sight of the observed person 915 to the observed person's pointof sight measuring section 214, and supplies the image used formeasuring the position at which the observed person 915 is present tothe observed person's position measuring section 212. Furthermore, theimage-capturing section 200 supplies the image used for measuring thepoint of sight of the observer 910 to the observer's point of sightmeasuring section 210. The observer's point of sight measuring section210 measures the point of sight of the observer 910 when theimage-capturing section 200 captures the image of the observed person915. In addition, the observer's point of sight measuring section 210further measures how the observer 910 directs the line of sight towardsthe observed person 915 when the image-capturing section 200 capturesthe image of the observed person 915. Specifically speaking, theobserver's point of sight measuring section 210 relates the point ofsight of the observer 910 to a time of measuring the point of sight ofthe observer 910, to measure the trajectory of the point of sight of theobserver 910. The observer's point of sight measuring section 210further measures the movement rate of the point of sight of the observer910 when the image-capturing section 200 captures the image of theobserved person 915.

To be specific, the observer's point of sight measuring section 210 maymeasure the point of sight of the observer 910 at predetermined timeintervals. For example, the observer's point of sight measuring section210 measures the point of sight of the observer 910 at a first time andthen at a second time. The observer's point of sight measuring section210 measures the movement rate of the point of sight based on a timeperiod calculated by using the first and second times and a distancecalculated by using the points of sight measured at the first and secondtimes. The observer's point of sight measuring section 210 supplies thetrajectory and movement rate of the point of sight of the observer 910to the observer's point of sight judging section 250 and point of sightmoving method learning section 216.

The observed person's position measuring section 212 measures an area,within a predetermined image-capturing target area of theimage-capturing section 200, in which the observed person 915 ispresent. Here, the observed person's position measuring section 212obtains the position at which the observed person 915 is present in theform of coordinate data defined in accordance with a predeterminedthree-dimensional coordinate system within the image-capturing targetarea. The observed person's position measuring section 212 relating tothe present embodiment measures an area corresponding to the face of theobserved person 915. The observed person's position measuring section212 may also measure an area corresponding to the eyes of the observedperson 915. For example, the observed person's position measuringsection 212 extracts a region of the face of the observed person 915based on edge detection and skin color extraction from the imagecaptured by the image-capturing section 200. The observed person'sposition measuring section 212 then measures the area corresponding tothe eyes of the observed person 915, which is included in the extractedregion of the face of the observed person 915. The observed person'sposition measuring section 212 supplies the coordinate data indicatingthe area in which the observed person 915 is present and the coordinatedata indicating the area corresponding to the eyes, to the observer'spoint of sight judging section 250.

The observed person's point of sight measuring section 214 measures thepoint of sight of the observed person 915. To be specific, the observedperson's point of sight measuring section 214 may include therein afirst observed person's line of sight measuring section for measuringthe line of sight of the left eye of the observed person 915 and asecond observed person's line of sight measuring section for measuringthe line of sight of the right eye of the observed person 915. The firstand second observed person's line of sight measuring sections measurethe directions of the lines of sight of the left and right eyes of theobserved person 915. For example, the first and second observed person'sline of sight measuring sections may calculate the lines of sight of theleft and right eyes of the observed person 915 in the same manner as thefirst and second observer's line of sight measuring sections. Here, bydefining a three-dimensional coordinate system in advance, coordinatedata can be generated which indicates the directions in which the linesof sight of the observed person 915 measured by the observed person'spoint of sight measuring section 214 are directed with respect to theobserved person 915.

The image-capturing unit 20 may further include therein the observer'sposition measuring section 270 for measuring the position at which theobserver 910 is present within the predetermined image-capturing targetarea of the image-capturing section 200. The observer's positionmeasuring section 270 measures the position, within the image-capturingtarget area, at which the observer 910 is present. The observer'sposition measuring section 270 obtains coordinate data that indicatesthe position at which the observer 910 is present in accordance with thethree-dimensional coordinate system defined in advance within theimage-capturing target area. The observer's position measuring section270 may determine both positions of the observer 910 and observed person915 in the same coordinate system. The observer's position measuringsection 270 obtains a plurality of pieces of coordinate data included inthe area in which the observer 910 is present, in order to identify thearea occupied by the observer 910.

The observed person's point of sight measuring section 214 measures thearea of the observer 910 at which the point of sight of the observedperson 915 is present, based on the coordinate data indicating the areawithin which the observer 910 is present which is obtained by theobserver's position measuring section 270 and the directions of thelines of sight of the observed person 915 which are measured by theobserved person's point of sight measuring section 214. To be specific,the observed person's point of sight measuring section 214 judges, asthe point of sight of the observed person 915, an intersecting pointbetween the area occupied by the observer 910 which is identified by thepieces of coordinate data which are obtained by the observer's positionmeasuring section 270 and an extended line of the line of sight of theobserved person 915 which is measured by the observed person's point ofsight measuring section 214. The observed person's point of sightmeasuring section 214 obtains the coordinate data indicating theintersecting point as coordinate data indicating the point of sight ofthe observed person 915.

Similarly to the observer's point of sight measuring section 210, theobserved person's point of sight measuring section 214 may includetherein an observed person's characteristic point extracting section forextracting characteristic points of the eyes and the like of theobserved person 915. The observed person's characteristic pointextracting section extracts the characteristic points of the observedperson 915. When the observed person 915 moves the head, the observedperson's point of sight measuring section 214 may trace the extractedcharacteristic points of the eyes and the like so as to follow themovement of the observed person 915, and measure the line of sight ofthe observed person 915. To be specific, the observed person's point ofsight measuring section 214 may include therein a movement controlsection for moving the image-capturing section 200 that captures imagesof the observed person 915 in accordance with the movement of thecharacteristic points of the observed person 915 which are extracted bythe observed person's characteristic point extracting section. Theobserved person's point of sight measuring section 214 suppliesinformation indicating the measured point of sight of the observedperson 915 to the observer's point of sight judging section 250.

The point of sight moving method learning section 216 learns how theobserver 910 directs the line of sight towards the observed person 915,and supplies information indicating the learned method of how to directthe line of sight of the observer 910 towards the observed person 915,to the point of sight moving method storing section 255. According tothe present embodiment, the point of sight moving method learningsection 216 learns how the observer 910 habitually moves the line ofsight based on the sound made when the observer 910 asks a question tothe observed person 915 and the trajectory of the movement of the pointof sight of the observer 910. The operation performed by the point ofsight moving method learning section 216 is described later in detail.

The point of sight moving method storing section 255 stores thereon apredetermined method of how the observer 910 directs the line of sighttowards the observed person 915. When there are a plurality ofobservers, the point of sight moving method storing section 255 maystore thereon one or more methods of how each observer directs the lineof sight. The point of sight moving method storing section 255 storesthereon the information indicating the method of how to direct the lineof sight which is received from the point of sight moving methodlearning section 216 in association with the observer. The point ofsight moving method storing section 255 supplies the informationindicating the method of how the observer 910 directs the line of sight,to the observer's point of sight judging section 250. The observer'spoint of sight judging section 250 judges whether the point of sight ofthe observer 910 which is received from the observer's point of sightmeasuring section 210 is present within a predetermined area. To bespecific, the observer's point of sight judging section 250 judgeswhether the point of sight of the observer 910 is present within thearea in which the observed person 915 is present.

The observed person's position measuring section 212 may obtain, as thearea in which the observed person 915 is present, a plurality of piecesof coordinate data determined in accordance with the predeterminedthree-dimensional coordinate system for the image-capturing target areaof the image-capturing section 200, for example. By comparing thedirection of the line of sight of the observer 910 with the coordinatedata indicating the area in which the observed person 915 is present,the observer's point of sight judging section 250 may make the judgmentwhether the point of sight of the observer 910 is present within thearea in which the observed person 915 is present. The observer's pointof sight judging section 250 may also judge whether the point of sightof the observer 910 is present within the area corresponding to the eyesof the observed person 915. Also, the observer's point of sight judgingsection 250 further judges whether the trajectory of the point of sightof the observer 910 which is received from the observer's point of sightmeasuring section 210 coincides with the trajectory of the point ofsight indicated by the information indicating the method of how todirect the line of sight which is received from the point of sightmoving method storing section 255. The observer's point of sight judgingsection 250 may additionally judge whether the movement rate of thepoint of sight of the observer 910 which is measured by the observer'spoint of sight measuring section 210 falls within a predetermined range.

The observer's point of sight judging section 250 judges whether thelines of sight of the observer 910 and observed person 915 coincide witheach other based on the point of sight of the observed person 915 whichis received from the observed person's point of sight measuring section214 and the point of sight of the observer 910 which is received fromthe observer's point of sight measuring section 210. For example, theobserver's point of sight judging section 250 may judge that the linesof sight of the observer 910 and observed person 915 coincide with eachother if the point of sight of the observer 910 which is measured by theobserver's point of sight measuring section 210 is present within thearea corresponding to the eyes of the observed person 915 and the pointof sight of the observed person 915 which is measured by the observedperson's point of sight measuring section 214 is present within the areacorresponding to the eyes of the observer 910. The observer's point ofsight judging section 250 supplies the result of the judgment to theimage storing control section 260.

When receiving from the observer's point of sight judging section 250 aresult of judgment indicating that the point of sight of the observer910 is present within the predetermined area, the image storing controlsection 260 obtains the image of the observed person 915 from theimage-capturing section 200, and stores the image onto the image storingsection 320. Here, the predetermined area may be the area within whichthe observed person 915 is present, the area corresponding to the eyesof the observed person 915, or the like. Here, the image storing controlsection 260 may store, onto the image storing section 320, one or moreimages of the observed person 915 which are captured by theimage-capturing section 200 during a predetermined time period startingfrom a time that precedes, by a predetermined time period, the time atwhich the observer's point of sight judging section 250 makes thejudgment that the point of sight of the observer 910 is present withinthe predetermined area. When the observer's point of sight judgingsection 250 judges that the method of how the observer 910 directs theline of sight towards the observed person 915 coincides with the methodof how to direct the line of sight which is stored on the point of sightmoving method storing section 255, the image storing control section 260may obtain the image of the observed person 915 from the image-capturingsection 200 and stores the obtained image onto the image storing section320.

Also, when the observer's point of sight judging section 250 judges thatthe movement rate of the line of sight of the observer 910 falls withinthe predetermined range, the image storing control section 260 mayobtain the image of the observed person 915 from the image-capturingsection 200 and store the obtained image onto the image storing section320. The image storing control section 260 may store, onto the imagestoring section 320, a plurality of images of the observed person 915which are captured by the image-capturing section 200 before and afterthe time at which the observer's point of sight judging section 250makes the judgment that the lines of sight of the observer 910 andobserved person 915 coincide with each other. In addition, the imagestoring control section 260 may detect the sound that is made when theobserver 910 asks a question to the observed person 915, obtain from theimage-capturing section 200 the image of the observed person 915 whichis captured by the image-capturing section 200 when the sound isdetected, and store the obtained image onto the image storing section320.

The image recording apparatus 10 relating to the present embodiment canautomatically record an image showing an area of the region of theobserved person 915 to which the observer 910 pays attention, at atiming at which the observer 910 pays attention to the observed person915, while the point of sight of the observer 910 is present within thepredetermined area, that is to say, while the observer 910 is observingthe observed person 915. This configuration makes it possible to easilyreproduce, at a later time than the observation, the image showing anarea of the observed person 915 which is captured when the observer 910pays attention to the observed person 915 while observing the observedperson 915.

Also, the image recording apparatus 10 relating to the presentembodiment can record the image of the observed person 915 if themovement rate of the point of sight of the observer 910 falls within thepredetermined range. In other words, the image recording apparatus 10does not record thereon the image of the observed person 915 when theobserver 910 does not pay attention to the observed person 915, forexample, when the movement rate of the point of sight of the observer910 is higher than a predetermined rate. Consequently, while theobserver 910 observes the observed person 915, the image recordingapparatus 10 can only record appropriate images of the observed person915 which are captured when the observer 910 pays attention to theobserved person 915.

In addition, the image recording apparatus 10 relating to the presentembodiment can record the image of the observed person 915 when thepoints of sight of the observer 910 and observed person 915 coincidewith each other. With this configuration, the image recording apparatus10 relating to the present embodiment can reproduce the image of theobserved person 915 which is captured when the points of sight of theobserver 910 and observed person 915 coincide with each other, togetherwith the sound information of the observer 910 in relation to a healthinterview, for example. Therefore, the image recording apparatus 10relating to the present embodiment can easily reproduce the facialexpressions presented by the observed person 915 in response to thequestions asked by the observer 910.

FIGS. 13, 14 and 15 are used to illustrate the operation performed bythe point of sight moving method learning section 216. FIG. 13schematically illustrates trajectories 835, 836 and 837 of the point ofsight which are obtained when different observers 910, 911 and 912observe the observed person 915. As seen from FIG. 13, each observer hasa different method of directing the line of sight towards the observedperson 915. Therefore, the point of sight moving method learning section216 identifies a unique method of moving the point of sight for eachobserver. To be specific, the point of sight moving method learningsection 216 relating to the present embodiment identifies a uniquemethod of moving the point of sight for each observer, by identifying aregion in which the point of sight of the observer is likely to movewhile the observer is talking.

The point of sight moving method learning section 216 receives, from theobserver's point of sight measuring section 210, the informationindicating the point of sight of the observer 910 at predetermined timeintervals. Also, the point of sight moving method learning section 216receives from the sound unit 40 information indicating whether there isa sound, and when a sound is detected, identifies the region in whichthe point of sight of the observer is present. Here, the point of sightmoving method learning section 216 may identify whether the point ofsight of the observer is present in a predetermined region within theimage of the observed person 915. In the present embodiment, forexample, a plurality of regions are defined in advance within the imageof the observed person 915, as shown in FIG. 13. For example, suchregions include a region A (the eyes), a region B (the mouth), a regionC (the hands), a region D (a portion of the image which shows theobserved person from the neck up but excludes the face), a region E (thebody), and a region F (the remaining portion). The point of sight movingmethod learning section 216 identifies the region in which the point ofsight of the observer is present, in association with the informationindicating whether there is a speech made by the observer. For eachregion, the point of sight moving method learning section 216 calculatesthe total time period for which the point of sight is present in theregion. Based on this operation, the point of sight moving methodlearning section 216 identifies the region in which the point of sightis present for the longest period of time when the speech is made. Inthe case of the observer 910 shown in FIGS. 13 and 14, for example, thepoint of sight is present in the region D when the observer 910 does notmake a speech, but the point of sight moves to the region correspondingto the eyes of the observed person (region A) when the observer 910makes a speech. In the case of the observer 911, the point of sightmoves to the region corresponding to the hands of the observed person(region C).

The point of sight moving method learning section 216 may also identifythe region in which the point of sight is present for the longest timeperiod when there is no speech made, by calculating the total timeperiod for which the point of sight is present for each region.Alternatively, the point of sight moving method learning section 216 maycalculate the likelihood at which the point of sight is present for eachpredetermined region. In the above-described manner, the point of sightmoving method leaning section 216 learns how each observer directs theline of sight, and the point of sight moving method storing section 255stores thereon the information regarding the learned method of how todirect the line of sight.

Here, the point of sight moving method learning section 216 may vary thesizes of the predetermined regions in the image of the observed person915 in accordance with the learned methods of how to direct the line ofsight. For example, the point of sight moving method learning section216 varies the size of each predetermined region in the image of theobserved person 915 in accordance with the period of time for which thepoint of sight is present in the region when the speech is made. To bespecific, in the case of a predetermined region in which the point ofsight is present for a long time period when a speech is made, the pointof sight moving method learning section 216 reduces the size of thepredetermined region. On the other hand, in the case of a predeterminedregion in which the point of sight is present for a short time periodwhen a speech is made, the point of sight moving method learning section216 increases the size of the predetermined region. Each observeruniquely behaves in a habitual manner when making a speech, for example,tends to stare at a particular area or move the point of sightfrequently. According to the present embodiment, the sizes of thepredetermined regions defined in the image of the observed person 915are varied depending on the method of how the observer directs the lineof sight. Therefore, the present embodiment can prevent unnecessaryimages which are captured when the point of sight moves differently fromthe habitual manner from being stored.

As indicated by the trajectory 837 of the observer 912 shown in FIG. 13,some observers may habitually observe patients without focusing thepoint of sight on a certain predetermined region (in FIG. 13, the regionB). Taking such a habit into consideration, the point of sight movingmethod learning section 216 may identify the region in which the pointof sight of the observer is highly likely to be present, and it may bejudged that the methods of how to direct the line of sight coincide witheach other when the point of sight of the observer 910 is present in theidentified region.

In addition to the above-described factor, the point of sight movingmethod learning section 216 may use the movement rate to learn themethod of how the observer directs the line of sight. For example, thepoint of sight moving method learning section 216 calculates the averagemovement rate of the point of sight of the observer while the sound unit40 detects a sound. With this configuration, the point of sight movingmethod learning section 216 can know the habit of the observer 910exhibited when the observer 910 directs the line of sight towards theobserved person 915, for example, slowly moves the line of sight,quickly moves the line of sight, and stares at the observed person 915during observation. The point of sight moving method storing section 255stores thereon the average movement rate of the point of sightcalculated for each observer. Also, the point of sight moving methodlearning section 216 may calculate, for each observer, the averagemovement rate of the point of sight while no sound is detected.

The observer's point of sight judging section 250 judges that themethods of how to direct the lines of sight coincide with each other,when the point of sight of the observer 910 is present while theobserver 910 is making a speech, in the region to which the observer 910is most likely to move the point of sight while making a speech,according to the information stored in the point of sight moving methodstoring section 255. Alternatively, the observer's point of sightjudging section 250 may make the judgment that the methods of how todirect the lines of sight coincide with each other, when the point ofsight of the observer 910 is present, while the observer 910 is making aspeech, in the region to which the observer 910 is most likely to movethe point of sight during a speech, and when the movement rate of thepoint of sight of the observer 910 during a speech is substantiallyequal to the average movement rate of the observer 910 during a speech,according to the information stored on the point of sight moving methodstoring section 255.

Here, by using the image recording apparatus relating to the presentembodiment, the observer 910 may observe a plurality of observedpersons. When the image captured by the image-capturing section 200includes a plurality of observed persons, the image storing controlsection 260 extracts and stores the region indicating an observed personin which the point of sight of the observer 910 measured by theobserver's point of sight measuring section 210 is included. In thiscase, the observer's point of sight judging section 250 judges whetherthe point of sight of the observer 910 measured by the observer's pointof sight measuring section 210 is present within the regioncorresponding to the face of the observed person which is calculated bythe observed person's position measuring section 212. When theobserver's point of sight judging section 250 judges positively, theimage storing control section 260 may store the image of the observedperson 915 captured by the image-capturing section 200. Here, the imagestoring control section 260 may extract only a region showing theobserved person who has the point of sight of the observer 910 measuredby the observer's point of sight measuring section 210 and store theextracted region.

Similarly to the image recording apparatus 10 described with referenceto FIGS. 1 to 11, the observer 910 may remotely observe the observedperson 915. Also, a plurality of observers may observe a plurality ofobserved persons via a network. FIGS. 16A and 16B show the case wherethe image recording apparatus 10 is used for a television conference. Inthis case, the image recording apparatus relating to the presentembodiment can be used for a television conference. To be specific,image recording apparatuses are respectively provided for the observerside and observed persons side, and connected to each other via anetwork. The sound unit 40 provided in each image recording apparatustransmits sound to the other image recording apparatus, and the outputsection 50 provided in each image recording apparatus displays the imageshowing the other side on the display section.

The image-capturing unit 20 of the observer side includes therein theimage-capturing sections 202 and 208, observer's point of sightmeasuring section 210 and observer's position measuring section 270. Theimage-capturing unit 20 of the observed person side includes therein theimage-capturing section 204, observed person's position measuringsection 212 and observed person's point of sight measuring section 214.Here, the coordinate systems of the image-capturing sections 202 to 208may have a predetermined relation with each other. For example, therelation among the coordinate systems is calibrated before theobservation is started via a network. As described earlier, theobserver's point of sight measuring section 210 measures the points ofsight of the observers 910 and 912 when the observers 910 and 912observe the images of the observed persons 915 and 916 displayed on thedisplay section included in the output section 50 of the observer side.The observed person's point of sight measuring section 214 measures thepoints of sight of the observed persons 915 and 916 when the observedpersons 915 and 916 observe the images of the observers 910 and 912displayed on the display section included in the output section 50 ofthe observed person side. The observer's position measuring section 270measures the position of the observer 910 within the predeterminedimage-capturing target area. The observed person's position measuringsection 212 measures the position of the observed person 915 within thepredetermined image-capturing target area. Except for these operations,the above-mentioned constituents operate as described earlier.

Even when connected to each other via a network, the plurality of imagerecording apparatuses can accurately record the change in the facialexpressions of the other side during conversation. In particular, when aspeaker is making a speech to a plurality of persons, the imagerecording apparatuses can accurately record the facial expressions of aperson to whom the speaker is paying attention during the speech.

FIG. 17 shows exemplary functional configurations of the image-capturingunit 20 relating to a third embodiment and the input unit 60 relating toa second embodiment. The image-capturing unit 20 relating to the thirdembodiment includes therein the image-capturing section 200, observer'spoint of sight measuring section 210, observed person's positionmeasuring section 212, observed person's point of sight measuringsection 214 and observer's point of sight judging section 250. The inputunit 60 relating to the second embodiment includes therein a line ofsight information input section 600. These configurations aim to providean image recording apparatus which makes it possible to, at a latertime, easily retrieve an image of the observed person 915 to which theobserver 910 pays attention. The image-capturing unit 20 and input unit60 relating to the present embodiment may additionally include thereinsome or all of the functions and configurations of the units whichtogether constitute the image recording apparatus 10 described withreference to FIGS. 1 to 11. The constituents assigned with the samereference numerals as in FIGS. 1 to 11 have substantially the samefunctions as the corresponding constituents in FIGS. 1 to 11, andtherefore not explained in detail in the following.

As mentioned in the description of the image-capturing unit 20 relatingto the first and second embodiments, when the image-capturing section200 is an omnidirectional camera, the image-capturing section 200 mayinclude the functions of the image-capturing sections 202 to 208.Alternatively, the image-capturing unit 20 may include, in place of theimage-capturing section 200, the image-capturing section 202 fordetecting the direction of the line of sight of the observer 910, theimage-capturing section 208 for measuring the area within which theobserver 910 is present, the image-capturing section 204 for detectingthe direction of the line of sight of the observed person 915, and theimage-capturing section 206 for measuring the area within which theobserved person 915 is present.

The image-capturing section 200 captures the image of the observedperson 915, and supplies the captured image of the observed person 915to the image storing section 320. The image-capturing section 200supplies the image used to measure the point of sight of the observedperson 915 to the observed person's point of sight measuring section214, and supplies the image used to measure the position at which theobserved person 915 is present to the observed person's positionmeasuring section 212. Also, the image-capturing section 200 suppliesthe image used to measure the point of sight of the observer 910 to theobserver's point of sight measuring section 210. The observed person'sposition measuring section 212 measures the area, within thepredetermined image-capturing target area of the image-capturing section200, in which the observed person 915 is present. The observed person'sposition measuring section 212 measures the area corresponding to theeyes of the observed person 915. The observed person's positionmeasuring section 212 supplies, to the observer's point of sight judgingsection 250, the information indicating the measured area within whichthe observed person 915 is present and the information indicating themeasured area corresponding to the eyes of the observed person 915.

The observer's point of sight measuring section 210 measures the pointof sight of the observer 910 when the image-capturing section 200captures the image of the observed person 915. The observer's point ofsight measuring section 210 also measures the method of how the observer910 directs the line of sight towards the observed person 915 when theimage-capturing section 200 captures the image of the observed person915. The observer's point of sight measuring section 210 also measuresthe movement rate of the point of sight of the observer 910 when theimage-capturing section 200 captures the image of the observed person915. The observer's point of sight measuring section 210 supplies themeasured information about the observer 910, such as the point of sight,the method of how to direct the line of sight, and the movement rate ofthe point of sight, to the observer's point of sight judging section 250and the image storing section 320.

The observed person's point of sight measuring section 214 measures thepoint of sight of the observed person 915. The observed person's pointof sight measuring section 214 may measure the point of sight of theobserved person 915 in association with the time, to measure informationindicating the trajectory of the point of sight of the observed person915. The observed person's point of sight measuring section 214 suppliesthe information indicating the measured point of sight of the observedperson 915 and the information indicating the trajectory of the point ofsight, to the observer's point of sight judging section 250. Theobserver's point of sight judging section 250 judges whether the pointof sight of the observer 910 measured by the observer's point of sightmeasuring section 210 is present within the area in which the observedperson 915 is present which is measured by the observed person'sposition measuring section 212. The observer's point of sight judgingsection 250 also judges whether the point of sight of the observer 910measured by the observer's point of sight measuring section 210 ispresent within the area corresponding to the face or hands of theobserved person 915 which is measured by the observed person's positionmeasuring section 212. Also, the observer's point of sight judgingsection 250 may judge whether the point of sight of the observer 910measured by the observer's point of sight measuring section 210 ispresent within the area corresponding to the eyes of the observed person915 which is measured by the observed person's position measuringsection 212. Furthermore, the observer's point of sight judging section250 judges whether the lines of sight of the observer 910 and observedperson 915 coincide with each other. The observer's point of sightjudging section 250 supplies the result of the judgment to the imagestoring section 320.

The image storing section 320 stores thereon the image captured by theimage-capturing section 200 in association with the informationindicating the point of sight of the observer 910, the method of how theobserver 910 directs the line of sight towards the observed person 915,the movement rate of the point of sight of the observer 910, and theinformation indicating the result of the judgment made by the observer'spoint of sight judging section 250. The image storing section 320supplies the stored image to the output section 50 under the control ofthe line of sight information input section 600. Here, the line of sightinformation input section 600 enables the user 917 to input informationindicating the point of sight of the observer 910. Specificallyspeaking, the line of sight information input section 600 enables theuser 917 to input information indicating that the point of sight of theobserver 910 is present within the area in which the observed person 915is present or information indicating that the point of sight of theobserver 910 is present within the area corresponding to the face orhands of the observed person 915. Also, the line of sight informationinput section 600 may enable the user 917 to input informationindicating that the point of sight of the observer 910 is present withinthe area corresponding to the eyes of the observed person 915. Here, thearea corresponding to the face, the area corresponding to the hands, andthe area corresponding to the eyes respectively may be regions ofpredetermined sizes including the face, hands and eyes and thesurrounding areas thereof. For example, the area corresponding to theeyes is a region defined by a circle that has a center at the black partof at least one of the left and right eyes and a radius of 1 cm.Alternatively, the area corresponding to the eyes may be a region whichincludes both of the eyes and is defined by a rectangle or ellipse thathas the long side in the horizontal direction of the face. The line ofsight information input section 600 may enable the user 917 to inputinformation indicating the method of how the observer 910 directs theline of sight towards the observed person 915, information indicatingthe movement rate of the point of sight, and information indicating thatthe lines of sight of the observer 910 and observed person 915 coincidewith each other. For example, when the user 917 desires to look at theimage of the observed person 915 which is captured when the observer 910pays attention to the eyes of the observed person 915, the user 917inputs, into the line of sight information input section 600,information indicating that the point of sight of the observer 910 ispresent at the eyes of the observed person 915. The line of sightinformation input section 600 supplies the information input by the user917 to the output section 50.

The output section 50 outputs the image of the observed person 915 whichis stored on the image storing section 320 in association with theinformation indicating the point of sight of the observer 910 which isinput through the line of sight information input section 600.Specifically speaking, the output section 50 receives the informationindicating the point of sight of the observer 910 input through the lineof sight information input section 600, and extracts and outputs theimage which is stored on the image storing section 320 in associationwith the information coinciding with the information received form theline of sight information input section 600.

The output section 50 may include therein an advising section. Theadvising section advises that the observer 910 should ask a differentquestion to the observed person 915, based on the change in the point ofsight of the observed person 915. Specifically speaking, the advisingsection judges whether the observed person 915 diverts the eyes from theobserver 910, based on the trajectory of the point of sight of theobserved person 915. When judging that the observed person 915 divertsthe eyes from the observer 910, the advising section displays on theoutput section 50 a character string advising that the observer 910should ask a different question to the observed person 915. Here, theoutput section 50 may advise the observer 910 to ask a differentquestion by way of sound.

According to the present embodiment, the image recording apparatus 10can store thereon the image of the observed person 915 in associationwith the information obtained when the observer 910 observes theobserved person 915, such as the point of sight of the observer 910, themethod of how the observer 910 directs the line of sight towards theobserved person 915, the movement rate of the point of sight, andinformation indicating the lines of sight of the observer 910 andobserved person 915 coincide with each other. Therefore, the imagerecording apparatus 10 can reproduce how the observer 910 observes theobserved person 915, at a later time than the actual observation of theobserved person 915 by the observer 910. In addition, the imagerecording apparatus 10 makes it possible to easily retrieve the image ofthe observed person 915 to which the observer 910 pays attention.

According to the present embodiment, the image storing section 320 maystore thereon the image of the observed person 915 captured by theimage-capturing section 200 further in association with informationindicating the point of sight of the observed person 915 which ismeasured by the observed person's point of sight measuring section 214.In this case, the line of sight information input section 600 furtherenables the user to input information indicating the point of sight ofthe observed person 915. The output section 50 outputs the image of theobserved person or observer which is stored on the image storing section320, based on the information indicating the point of sight of theobserved person 915 and the information indicating the point of sight ofthe observer 910 which are input through the line of sight informationinput section 600. With this configuration, the image recordingapparatus 10 relating to the present embodiment makes it possible toreproduce the image of the observed person at a later time than theobservation, based on the combination of the points of the sight of theobserver and observed person.

FIGS. 18 and 19 are used to illustrate the operation performed by theobserver's point of sight judging section 250. FIG. 20 shows one exampleof the data stored on the image storing section 320. FIG. 18A shows aplurality of predetermined regions 850 to 858 in the image of theobserved person 915, and FIG. 18B shows a plurality of predeterminedregions 860 to 868 in the image of the observer 910. The regions 850,852 and 854 are, for example, the areas corresponding to the face, eyesand mouth of the observed person 915. The regions 856 and 858 are theareas corresponding to the hands (right and left hands) of the observedperson 915. The regions 860 and 862 are the areas corresponding to theeyes (left and right eyes) of the observer 910, and the region 864 isthe area corresponding to the mouth of the observer 910. The regions866, 868 and 870 are respectively right, left and upper regions, withrespect to the observer 910, which do not include the observer 910.

The observer's point of sight judging section 250 judges whether thepoint of sight of the observer 910 is present in any one of the regions850 to 858 while the observer 910 is observing the observed person 915.Also, the observer's point of sight judging section 250 judges whetherthe point of sight of the observed person 915 is present in any one ofthe regions 860 to 868. In the present embodiment, the observer's pointof sight judging section 250 receives pieces of coordinate dataindicating the points of sight from the observer's point of sightmeasuring section 210 and observed person's point of sight measuringsection 214, as pieces of information indicating the points of sight ofthe observer 910 and observed person 915, and judges whether thereceived points of sight are included in any of the predeterminedregions, as shown in FIG. 19. The image storing section 320 storesthereon the result of the judgment in accordance with the data structureshown in FIG. 20, for example. According to the example shown in FIG.20, the image storing section 320 stores thereon the coordinate dataindicating the point of sight of the observer and the region in theimage of the observed person 915 which includes the point of sight ofthe observer, in association with the image of the observed person theobserver looks at. The image storing section 320 also stores thereon thecoordinate data indicating the point of sight of the observed person andthe region in the image of the observer 910 which includes the point ofsight of the observed person, in association with the image of theobserved person.

For example, the line of sight information input section 600 enables theuser to input, as the information indicating the point of sight of theobserver 910, the name of the region to which the observer 910 paysattention, and, as the information indicating the point of sight of theobserved person 915, the name of the region to which the observed person915 pays attention to. It is assumed that the observer is a medicaldoctor, for example. Here, if the medical doctor desires to retrieve animage which is captured when the medical doctor pays attention to thearea including the mouth of a patient who is the observed person, themedical doctor inputs a keyword of “mouth” as the information indicatingthe point of sight of the observer 910. In this case, the output section50 outputs a moving image 1 which is captured when the medical doctorpays attention to the mouth and surrounding area of the patient. Also,based on the information indicating the point of sight of the observerwhich is input through the line of sight information input section 600,the output section 50 may extract and output the predetermined regioncorresponding to the input point of sight from the image of the observedperson 915 stored on the image storing section 320. In this case, theoutput section 50 extracts and outputs, as the predetermined regioncorresponding to the input point of sight, a predetermined region in theimage of the observed person which is stored in association with theinformation indicating the point of sight of the observer 910. Forexample, when a keyword “right eye” is input as the informationindicating the point of sight of the observer, the image showing theregion 852 corresponding to the right eye of the patient is extractedfrom the moving image 1 which is stored in association with the keyword“right eye”, and the extracted image is output.

According to the present embodiment, the sound recording section 400 inthe sound unit 40 records sound information of the observer 910 andobserved person 915. Here, the image storing section 320 further storesthereon the sound information of the observer 910 which is recorded bythe sound recording section 400 in association with the image of theobserved person 915 which is captured by the image-capturing section200. The line of sight information input section 600 enables the user tofurther input the information indicating the sound information of theobserver 910. The output section 50 outputs the image of the observedperson 915 which is stored on the image storing section 320, based onthe input sound information of the observed person 915.

Note that the sound information is, for example, conversation exchangedwhen the observer 910 observes the observed person 915. The line ofsight information input section 600 stores, as the sound information,sound data and information regarding the contents of the sound data, inassociation with the image of the observed person 915. Here, the line ofsight information input section 600 may store a keyword or sentence, asthe information regarding the contents of the sound data. The line ofsight information input section 600 enables the user to input a keywordrelating to the contents of the observation performed by the observer910 on the observed person 915, as the information indicating the soundinformation of the observer 910. In this way, the image recordingapparatus relating to the present embodiment can reproduce how theobserver 910 observes the observed person 915 at a later time than theobservation. Also, the image recording apparatus relating to the presentembodiment can easily retrieve the image of the observed person 915based on the information indicating the area to which the observer 910pays attention to during the observation. Furthermore, the imagerecording apparatus relating to the present embodiment can retrieve adesired image based on the information regarding the conversationexchanged between the observer and observed person.

Here, the image-capturing section 200 relating to the present embodimentmay capture an image of a plurality of observed persons. In this case,an image is captured which indicates the area, within the predeterminedimage-capturing target area of the image-capturing section 200, in whichthe plurality of observed persons 915 are present, and the observedperson's position measuring section 212 supplies the informationindicating the area in which the plurality of observed persons 915 arepresent, to the observer's point of sight judging section 250. Theobserver's point of sight judging section 250 identifies an areacorresponding to one of the observed persons in which the point of sightof the observer measured by the observer's point of sight measuringsection 210 is present. Here, the observer's point of sight judgingsection 250 may judge whether the point of sight of the observer 910 ispresent in the area corresponding to the face, hands or eyes of theidentified observed person 915 which is measured by the observedperson's position measuring section 212. The image storing section 320stores thereon the image of the observed person who is identified, bythe observer's point of sight judging section 250, so as to correspondto the point of sight of the observer, in association with theinformation indicating the point of sight of the observer. The line ofsight information input section 600 enables the user to inputinformation indicating the point of sight of the observer, and theoutput section 50 outputs the image of the observed person which isstored on the image storing section 320 in association with the inputinformation indicating the point of sight of the observer.

The image recording apparatus 10 including therein the image-capturingunit 20 relating to the present embodiment may be used when a pluralityof observers observe a plurality of observed persons via a network. Forexample, the image recording apparatus 10 may be used for a televisionconference. In this case, in association with the point of sight of aspeaker on the observer side and the point of sight of the audience whoobserves the speaker on the display section on the observed person side,the image storing section 320 may store thereon the image of the speakerand the image of the audience listening to the speaker. Also, the imagestoring section 320 may store thereon the images of the speaker andaudience in association with the contents of the speech made by thespeaker. The line of sight information input section 600 enables theuser to input information indicating the point of sight of the speaker,the point sight of the audience, and/or contents of the speech made bythe speaker. The output section 50 outputs the images of the speaker andaudience which are stored in association with the input informationindicating the point of sight of the speaker, the point of sight of theaudience, and/or contents of the speech made by the speaker. In thisway, the image recording apparatus 10 relating to the present embodimentmakes it possible to retrieve an image by using the points of sight of aspeaker and audience as a keyword after the television conference.

FIG. 21 illustrates exemplary functional configurations of theimage-capturing unit 20 relating to a fourth embodiment and the inputunit 60 relating to a third embodiment included in the image recordingapparatus 10. The image-capturing unit 20 includes therein theimage-capturing section 200 and image storing control section 260. Thestoring unit 30 includes therein the image storing section 320 and anelectronic medical record storing section 330. The input unit 60includes therein an observation result input section 610, an inputdetecting section 620, an observation result information input section630, a time input section 640, and an observation result storing controlsection 650. Here, the observation result input section 610 includestherein the digital input section 612 and writing section 614. With theabove-described configurations, the image recording apparatus 10relating to the present embodiment aims to make it possible toappropriately reproduce the facial expressions of the observed person915 at a later time, even if the observer 910 can not see the facialexpressions of the observed person 915 while inputting the observationresult into an electronic medical record or the like. Here, the imagerecording apparatus 10 relating to the present embodiment may furtherinclude some or all of the functions and configurations of the unitswhich together constitute the image recording apparatus 10 describedwith reference to FIGS. 1 to 11. The constituents assigned with the samereference numerals as in FIGS. 1 to 11 have substantially the samefunctions as the corresponding constituents in FIGS. 1 to 11, andtherefore not explained in detail in the following.

The image-capturing section 200 captures an image of the observed person915, and supplies the captured image of the observed person 915 to theimage storing section 320. The image-capturing section 200 also suppliesthe image to the image storing control section 260. The electronicmedical record storing section 330 stores thereon en electronic medicalrecord to which a result of the observation on the observed person 915is input. Specifically speaking, the observation result input section610 is used to input the result of the observation done by the observer910. The observation result input section 610 supplies informationindicating the input of the observation result, to the input detectingsection 620. The observation result input section 610 then supplies theobservation result which is input by the observer 910 to the observationresult storing control section 650. The observation result storingcontrol section 650 stores, as an electronic medical record, theobservation result received from the observation result input section610 onto the electronic medical record storing section 330. To be morespecific, the digital input section 612 included in the observationresult input section 610 is used to input the result of the observationdone by the observer 910 into the electronic medical record. The digitalinput section 612 is, for example, a keyboard, a mouse, a touch panel orthe like. The digital input section 612 supplies, to the input detectingsection 620, the information indicating that the observer 910 isinputting the observation result into the electronic medical record.

The writing section 614 included in the observation result input section610 is used by the observer 910 to write the observation result into amedical record. The writing section 614 may include a case 615, apressing detecting section 616, a pen tip 617, and a detected resulttransmitting section 618. The pressing detecting section 616 detects achange in pressure which is caused by application of pressure to the pentip 617 when the observer 910 writes the observation result. Thepressing detecting section 616 supplies information indicating thedetected change in pressure to the detected result transmitting section618. The detected result transmitting section 618 supplies theinformation indicating the change in pressure to the input detectingsection 620. The detected result transmitting section 618 may supply theinformation indicating the change in pressure to the input detectingsection 620 by way of wired or wireless communication.

The input detecting section 620 detects that the digital input section612 is used to input the result of the observation done by the observer910 into the electronic medical record stored on the electronic medicalrecord storing section 330. Also, the input detecting section 620detects that the writing section 614 is used to write the result of theobservation done by the observer 910 into the medical record. Note thatthe writing section 614 may be used to write the observation result intoa paper medical record. The writing section 614 may be a pen or the likefor a digital white board. When the writing section 614 is a pen for adigital white board, the writing section 614 can be used to write theobservation result into the electronic medical record via the digitalwhite board. In this case, the observation result written by using thewriting section 614 is supplied to the observation result storingcontrol section 650.

The image storing control section 260 stores the image of the observedperson 915 captured by the image-capturing section 200 onto the imagestoring section 320, when the input detecting section 620 detects thatthe result of the observation done by the observer 910 is input. To bespecific, the image storing control section 260 obtains the image of theobserved person 915 captured by the image-capturing section 200, whenreceiving, from the input detecting section 620, the informationindicating that the result of the observation done by the observer 910is input by using the observation result input section 610. The imagestoring control section 260 then stores the obtained image of theobserved person 915 onto the image storing section 320. Alternatively,the image storing control section 260 stores, onto the image storingsection 320, images of the observed person 915 which are captured by theimage-capturing section 200 for a predetermined period of time startingfrom a time that precedes, by a predetermined time period, the time atwhich the input detecting section 620 detects that the result of theobservation done by the observer 910 is input. The image storing section320 stores the image captured by the image-capturing section 200 inassociation with information indicating the result of the observationdone by the observer 910 which is input by means of the observationresult input section 610 and the time at which the observer 910 inputsthe observation result. Here, the information indicating the result ofthe observation may include the name of a disease, a symptom,progression of the symptom, and improvement of the symptom. The imagestoring section 320 supplies the image stored thereon to the outputsection 50 under the control of the observation result information inputsection 630 and time input section 640.

The observation result information input section 630 enables the user917 to input information indicating a result of observation done by theobserver 910. Here, the user 917 may be the observer 910, a differentobserver than the observer 910, or the observed person 915, for example.When the user 917 desires to know what facial expression of the observerperson 915 helps the observer 910 judge the symptom of the observedperson 915, for example, the user 917 inputs, as the informationindicating the observation result, the name of the symptom observed bythe observer 910 into the observation result information input section630. The observation result information input section 630 causes theoutput section 50 to output an image of the observed person 915 which isstored on the image storing section 320 in association with the name ofthe symptom input by the user 917. The time input section 640 enablesthe user 917 to input a time. For example, when the user 917 desires tolook at a previous image of the observed person 915, the user 917 inputsa time corresponding to a desired image through the time input section640. The time input section 640 causes the output section 50 to outputan image of the observed person 915 which is stored on the image storingsection 320 in association with the time input by the user 917.

The output section 50 outputs an image of the observed person 915 whichis stored on the image storing section 320 in association with theinformation indicating the result of the observation done by theobserver 910 which is input through the observation result informationinput section 630. Also, the output section 50 outputs an image of theobserved person 915 which is stored on the image storing section 320 inassociation with the time input through the time input section 640. Theoutput section 50 may be a display apparatus such as a display screen.The output section 50 may display, side by side, images of the observedperson 915 in the chronological order. Alternatively, the output section50 may display, one at a time, images of the observed person 915 in thechronological order at predetermined time intervals.

The output section 50 may be provided in a head-mounted display. Whenthe output section 50 is a head-mounted display, the observer 910 canlook at, by way of the output section 50 provided in the head-mounteddisplay, the images of the observed person 915 even while, for example,inputting the observation result into an electronic or paper medicalrecord. In this case, the output section 50 may display the images ofthe observed person 915 at an area which does not make it difficult forthe observer 910 to input the observation result into the electronicmedical record or the like. For example, the head-mounted display mayinclude therein an observer's line of sight detecting section fordetecting the line of sight of the observer 910. The output section 50may display the images in a different area which is selected dependingon the line of sight of the observer 910 which is detected by theobserver's line of sight detecting section. The output section 50 maynot be required to display the images of the observed person 915 on thehead-mounted display when the observer 910 faces the observed person915. Here, the image-capturing section 200 may be provided at theentrance and exit of a clinic, for example, so as to capture images ofthe facial expressions exhibited by the observed person 915 when theobserved person 915 walks into and out the clinic.

While the observer 910 is inputting or writing the result of observingthe observed person 915 into an electronic or paper medical record, theimage recording apparatus 10 relating to the present embodiment detectsthe timing of inputting and writing by the observer 910 andautomatically captures an image of the observed person 915 at thedetected timing. In this way, it is made possible to reproduce, at alater time, the facial expressions of the observed person 915 which arenot actually observed by the observer 910 because those facialexpressions are exhibited while the observer 910 inputs the observationresult into an electronic medical record or the like.

Also the image recording apparatus 10 relating to the present embodimentcan obtain images of the observed person 915 which are captured for apredetermined period of time starting from a time that precedes, by apredetermined time period, the time at which the observer 910 inputs theresult of observing the observed person 915 into an electronic medicalrecord, so as to enable the observer 910 to look at the missed images ofthe observed person 915 in the chronological order. In this way, theobserver 910 can be reliably prevented from missing an image showing theobserved person 915 which may have an influence on the observationresult of the observed person 915.

FIG. 22 illustrates one example of the writing section 614 relating tothe present embodiment. The writing section 614 includes the case 615,pressing detecting section 616, pen tip 617, and detected resulttransmitting section 618. The pressing detecting section 616 detectspressure applied to the pen tip 617 when the pressure is applied to thepen tip 617. For example, the pressing detecting section 616 detectspressure applied to the pen tip 617 when the observer 910 writes aletter or the like on paper 390. The pressing detecting section 616supplies information indicating the detected pressure to the detectedresult transmitting section 618. The detected result transmittingsection 618 supplies the information indicating the pressure detected bythe pressing detecting section 616 to the input detecting section 620.The detected result transmitting section 618 and input detecting section620 may be coupled to each other by wired or wireless connection.

FIG. 23 illustrates exemplary data stored on the image storing section320 relating to the present embodiment. According to the example shownin FIG. 23, the image storing section 320 stores thereon a moving imagewhich is captured by the image-capturing section 200 for a period oftime starting at a time that precedes, by three minutes, the time atwhich the observation result is input and ending at the time at whichthe observation result is input. Note that the sound recording section400 included in the sound unit 40 records sound information of theobserver 910 while the image-capturing section 200 captures an image ofthe observed person 915. Here, the sound information is conversationexchanged when the observer 910 observes the observed person 915, forexample. The line of sight information input section 600 stores, as thesound information, sound data and information regarding the contents ofthe sound data in association with an image of the observed person 915.In this case, the observation result input section 610 may store, as theinformation regarding the contents of the sound data, a keyword orsentence. The image storing section 320 stores thereon the images of theobserved person 915 which are captured by the image-capturing section200 further in association with the sound information of the observer910 which is recorded by the sound recording section 400. Theobservation result input section 610 may enable the user to input akeyword relating to the contents of the observation on the observedperson 915 by the observer 910, and the output section 50 may output theimage of the observed person 915 which is stored on the image storingsection 320 in association with the sound information of the observer910.

FIGS. 24, 25A and 25B are used to illustrate an electronic medicalrecord 500 relating to the present embodiment. The output section 50displays, on a display section included in the output section 50, anelectronic medical record stored on the electronic medical recordstoring section 330. FIG. 24 illustrates an exemplary user interface forthe electronic medical record 500. The electronic medical record storingsection 330 relating to the present embodiment stores thereon theobservation result by way of an image 504 which schematicallyillustrates a patient body. The image 504 schematically illustrating thepatient body may be an image showing part of a human body such as anupper body, a lower body, limbs, and a face, or an image showing theentire human body.

The electronic medical record storing section 330 stores informationindicating whether a disease is found in association with predeterminedareas in the image 504 schematically illustrating a human body. Indetail, the electronic medical record 500 has checkboxes respectivelycorresponding to the predetermined areas of the image 504 schematicallyillustrating a human body. Here, the checkboxes are checked or cleareddepending of whether a disease is found by way of the digital inputsection 612. The electronic medical record 500 shown in FIG. 24 has aplurality of checkboxes 502 respectively corresponding to differentareas of the face of the patient. With this configuration, the observer910 can quickly complete inputting the result of the observation intothe electronic medical record.

Here, the image 504 schematically illustrating a patient may be replacedby an actual image of the observed person 915 which is captured by theimage-capturing section 200. If such is the case, the electronic medicalrecord 500 has checkboxes respectively corresponding to predeterminedareas of the actual image of the observed person 915 which is capturedby the image-capturing section 200. Alternatively, the electronicmedical record 500 may have checkboxes corresponding to at least apartial region of the image of the observed person 915 which is capturedby the image-capturing section 200. Here, the partial region of theimage of the observed person 915 may be a region indicating apredetermined area of a patient body, such as the face, eyes, mouth, andnose of the observed person 915. FIGS. 25A and 25B are used toillustrate exemplary electronic medical record which uses the actualimage of the observed person 915.

When one or more of the checkboxes are checked in accordance with theinput through the digital input section 612, the image storing section320 stores thereon the image of the observed person 915 which isassociated with the checkboxes, together with the information indicatingthe observation result. In this way, the image storing section 320relating to the present embodiment stores at least a partial region ofthe image of the observed person 915 which is captured by theimage-capturing section 200, in association with the informationindicating the observation result, based on the input by way of thedigital input section 612. Here, when the observation result is input bychecking the checkboxes, the image storing section 320 stores thereonsome of the images of the observed person 915 which are captured by theimage-capturing section 200 during a predetermined period of timestarting at a time that precedes, by a predetermined time period, thetime at which the input detecting section 620 detects the input of theobservation result. The output section 50 may display an image obtainedby enlarging at least part of the image of the observed person 915 whichis stored on the image storing section 320. Alternatively, the outputsection 50 may display at least part of each of the images of theobserved person 915 which are stored on the image storing section 320,in the chronological order within the electronic medical record.

As described above, the electronic medical record 500 relating to thepresent embodiment uses the actual image of the observed person 915,thereby enabling a change in symptoms to be accurately known. Also,since the electronic medical record 500 uses part of the image of theobserved person 915, the privacy of the observed person 915 can beprotected.

FIG. 26 illustrates an exemplary functional configuration of theimage-capturing unit 20 relating to a fifth embodiment. Theimage-capturing unit 20 includes therein the image-capturing section200, observer's point of sight measuring section 210, observed person'sposition measuring section 212, observed person's point of sightmeasuring section 214, observer's point of sight judging section 250,image storing control section 260, observer's position measuring section270, and a line of sight coincidence judging section 280. The followingdescription is made based on an assumption that the image recordingapparatus 10 relating to the present embodiment is used in the medicalfield. The image recording apparatus 10 relating to the presentembodiment quantifies the physical status of the observed person 915when the observer 910 observes the observed person 915, and recordsthereon the quantified physical status. Based on this, the imagerecording apparatus 10 relating to the present embodiment aims to outputinformation indicating a disease candidate which a person underobservation may possibly suffer from, when the person under observationis newly observed. Also, the image recording apparatus 10 aims tofacilitate observation by generating an image corresponding to adisease.

Here, when the image-capturing section 200 is an omnidirectional camera,for example, the image-capturing section 200 may include functions ofthe image-capturing section 202 for detecting the point of sight of theobserver 910, the image-capturing section 208 for measuring the area inwhich the observer 910 is present, the image-capturing section 204 fordetecting the point of sight of the observed person 915, and theimage-capturing section 206 for measuring the area in which the observedperson 915 is present. Alternatively, the image-capturing unit 20 mayinclude therein, in place of the image-capturing section 200, theimage-capturing section 202 for detecting the direction of the line ofsight of the observer 910, the image-capturing section 208 for measuringthe area in which the observer 910 is present, the image-capturingsection 204 for detecting the direction of the line of sight of theobserved person 915, and the image-capturing section 206 for measuringthe area in which the observed person 915 is present.

FIG. 27 illustrates an exemplary functional configuration of the inputunit 60 relating to a fourth embodiment. The input unit 60 relating tothe present embodiment includes therein an image input section 602, adisease information input section 604, the observation result inputsection 610, and the observation result storing control section 650.FIG. 28 illustrates an exemplary functional configuration of theanalyzing unit 70 relating to a second embodiment. The analyzing unit 70includes therein an observer's point of sight position judging section705, an image extracting section 710, a physical status quantifyingsection 715, a physical status data storing section 720, a physicalstatus calculating section 725, a quantified data extracting section730, a physical status comparing section 735, a likelihood calculatingsection 740, a disease image generating section 745, a likely diseaseoutput control section 750, and an image output control section 755.Here, the image recording apparatus 10 relating to the presentembodiment may further include some or all of the functions andconfigurations of the units which together constitute the imagerecording apparatus 10 described with reference to FIGS. 1 to 11. Theconstituents assigned with the same reference numerals as in FIGS. 1 to11 have substantially the same functions as the correspondingconstituents in FIGS. 1 to 11, and therefore not explained in detail inthe following.

The image-capturing section 200 captures an image of the observed person915. The image-capturing section 200 may capture a moving image of theobserved person 915. The image-capturing section 200 supplies thecaptured image to the image storing control section 260 under control ofthe image storing control section 260. Also, the image-capturing section200 supplies the image used to measure the point of sight of theobserved person 915 to the observed person's point of sight measuringsection 214, and the image used to measure the position at which theobserved person 915 is present to the observed person's positionmeasuring section 212. In addition, the image-capturing section 200supplies the image used to measure the point of sight of the observer910 to the observer's point of sight measuring section 210, and theimage used to measure the position at which the observer 910 is presentto the observer's position measuring section 270. The observer's pointof sight measuring section 210 measures the point of sight of theobserver 910 when the image-capturing section 200 captures the image ofthe observed person 915. The observer's point of sight measuring section210 supplies the measured point of sight to the observer's point ofsight judging section 250 and line of sight coincidence judging section280.

The observed person's point of sight measuring section 214 measures thepoint of sight of the observed person 915 when the image-capturingsection 200 captures the image of the observed person 915. The observedperson's point of sight measuring section 214 supplies the measuredpoint of sight to the observer's point of sight judging section 250 andline of sight coincidence judging section 280. The observer's positionmeasuring section 270 measures the position of the observer 910 withinthe predetermined image-capturing target area of the image-capturingsection 200. The observer's position measuring section 270 suppliesinformation indicating the measured position of the observer 910 to theobserver's point of sight judging section 250 and line of sightcoincidence judging section 280. The observed person's positionmeasuring section 212 measures the position at which the observed person915 is present within the image-capturing target area. The observedperson's position measuring section 212 supplies information indicatingthe measured position of the observed person 915 to the observer's pointof sight judging section 250 and line of sight coincidence judgingsection 280.

The observer's point of sight judging section 250 judges whether thepoint of sight of the observer 910 which is received from the observer'spoint of sight measuring section 210 is present within a predeterminedarea. Here, the predetermined area may be the area occupied by theobserved person 915, an area corresponding to the eyes of the observedperson 915 or the like. To be specific, the observer's point of sightjudging section 250 judges whether the point of sight of the observer910 is present within the predetermined area, based on the measurementresults received from the observer's point of sight measuring section210, observer's position measuring section 270, observed person's pointof sight measuring section 214, and observed person's position measuringsection 212. The observer's point of sight judging section 250 supplies,to the image storing section 320, information indicating the point ofsight of the observer 910 which is measured by the observer's point ofsight measuring section 210, and one of information indicating that thepoint of sight of the observer 910 is present within the predeterminedarea and information indicating that the point of sight of the observer910 is not present within the predetermined area.

The line of sight coincidence judging section 280 judges whether thelines of sight of the observer 910 and observed person 915 coincide witheach other based on the point of sight of the observed person 915 whichis measured by the observed person's point of sight measuring section214 and the point of sight of the observer 910 which is measured by theobserver's point of sight measuring section 210. Specifically speaking,the line of sight coincidence judging section 280 judges whether thepoint of sight of the observer 910 is present within the areacorresponding to the eyes of the observed person 915 based on themeasurement results supplied by the observed person's position measuringsection 212 and observer's point of sight measuring section 210. Forexample, the line of sight coincidence judging section 280 judgeswhether the coordinate indicating the point of sight of the observer 910is included in the area corresponding to the eyes of the observed person915 which is defined by a plurality of coordinates. In the same manner,the line of sight coincidence judging section 280 judges whether thepoint of sight of the observed person 915 is present within the areacorresponding to the eyes of the observer 910 based on the measurementresults supplied by the observer's position measuring section 270 andobserved person's point of sight measuring section 214. When the pointof sight of the observer 910 is present within the area corresponding tothe eyes of the observed person 915 and the point of sight of theobserved person 915 is present within the area corresponding to the eyesof the observer 910, the line of sight coincidence judging section 280judges that the lines of sight of the observer 910 and observed person915 coincide with each other. The line of sight coincidence judgingsection 280 supplies the result of the judgment to the image storingcontrol section 260. In this case, the line of sight coincidence judgingsection 280 may supply the judgment result to the image storing controlsection 260 as observed person's point of sight position information.

The image storing control section 260 obtains the image of the observedperson 915 from the image-capturing section 200, and supplies theobtained image to the image storing section 320. The image storingcontrol section 260 also supplies the judgment result from the line ofsight coincidence judging section 280 to the image storing section 320.The observation result input section 610 is used to input the result ofthe observation on the observed person 915 done by the observer 910.Here, the observation result may be a disease, and the disease inputthrough the observation result input section 610 is a name of a disease,for example. The observation result input section 610 may be used toinput information indicating the symptom of the disease observed by theobserver 910. When the observer 910 judges that the observed person 915does not suffer from a disease, the observation result input section 610is used to input information indicating that no disease is found, as theresult of the observation conducted on the observed person 915 by theobserver 910. The observation result input section 610 supplies theinput disease to the observation result storing control section 650. Theobservation result storing control section 650 supplies the diseaseinput through the observation result input section 610 to the imagestoring section 320.

The image input section 602 is used to input an image of a person underobservation. The image input section 602 may be used to input the imageof the observed person 915 which is captured by the image-capturingsection 200. Here, the person under observation may be an observedperson 915 who is to be newly observed by the observer 910. Meanwhile,the observed person 915 may be a person who has already been observed bythe observer 910 and whose image has already been stored on the imagestoring section 320. The image input section 602 may be used to input animage of the person under observation which is captured when the persondoes not suffer from a disease, and an image of the person underobservation which is obtained by the image-capturing section 200 whenthe observer 910 observes the new observed person 915. The image inputsection 602 supplies the input image of the person under observation tothe physical status calculating section 725. The disease informationinput section 604 is used to input information indicating a disease.Specifically speaking, the disease information input section 604 is usedby the observer 910 to input information indicating a disease of theobserved person 915. The disease information input section 604 suppliesthe information indicating the disease of the observed person 915 to thequantified data extracting section 730.

The image storing section 320 stores thereon the disease received fromthe observation result input section 610 and the observer's point ofsight position information indicating the point of sight of the observer910 which is measured by the observer's point of sight measuring section210, in association with the image of the observed person 915 which iscaptured by the image-capturing section 200. Here, the observer's pointof sight position information may indicate the result of the judgmentmade by the line of sight coincidence judging section 280, that is tosay, the observed person's point of sight position information, andwhether the observer 910 looks at the observed person 915. The imagestoring section 320 further stores thereon information indicating thatno disease is found which is input through the observation result inputsection 610 in association with the image of the observed person 915which is captured by the image-capturing section 200. The image storingsection 320 supplies the observer's point of sight position informationto the observer's point of sight position judging section 705. Also, theimage storing section 320 supplies the image of the observed person 915to the image extracting section 710 and physical status quantifyingsection 715.

Based on the observer's point of sight position information which isstored on the image storing section 320, the observer's point of sightposition judging section 705 judges whether the observer 910 looks atthe observed person 915 when the image of the observed person 915, whichis stored on the image storing section 320 in association with theobserver's point of sight position information, is captured. To bespecific, the observer's point of sight position judging section 705 mayjudge that the observer 910 looks at the observed person 915, if thepoint of sight of the observer 910 which is indicated by the observer'spoint of sight position information is present within the area of theobserved person 915. The observer's point of sight position judgingsection 705 supplies, to the image extracting section 710, theobserver's point of sight position information when judging that theobserver 910 looks at the observed person 915. When receiving, from theimage storing section 320, the observer's point of sight positioninformation indicating that the line of sight coincidence judgingsection 280 judges that the lines of sight of the observer 910 andobserved person 915 coincide with each other, the observer's point ofsight position judging section 705 supplies the received the observer'spoint of sight position information to the image extracting section 710.

The image extracting section 710 extracts an image of the observedperson 915 which is stored on the image storing section 320 inassociation with the observer's point of sight position informationreceived from the observer's point of sight position judging section705. The image extracting section 710 may extract a plurality of imagesof the observed person 915 which are stored on the image storing section320 in association with the observer's point of sight positioninformation. Specifically speaking, the image extracting section 710extracts an image of the observed person 915 which is captured at a timethat precedes, by a predetermined time period, the image-capturing timeat which the image of the observed person 915, which is stored on theimage storing section 320 in association with the observer's point ofsight position information, is captured, and an image of the observedperson 915 which is captured when a predetermined time period elapsesafter the image-capturing time. For example, the image extractingsection 710 may extract a plurality of images of the observed person 915which are captured within a predetermined period of time that is definedwith respect to the image-capturing time at which the image of theobserved person 915 which is stored in association with the observer'spoint of sight position information is captured. The image extractingsection 710 supplies the extracted images to the physical statusquantifying section 715.

The physical status quantifying section 715 analyzes the plurality ofimages of the observed person 915 extracted by the image extractingsection 710, for each of the diseases stored on the image storingsection 320 in association with the images of the observed person 915received from the image extracting section 710. The physical statusquantifying section 715 quantifies the physical status of the observedperson 915. Here, the physical status of the observed person 915 mayinclude information regarding the observed person 915, for example, thesize of the pupil, the point of sight, the number of blinks measuredwithin a predetermined time period, twitching of the eyelids or face,the color of the face, the temperature of the face, the sag of thecheeks, swelling of the face, and the condition of the skin (a rash andthe like). Specifically speaking, the physical status quantifyingsection 715 quantifies a change in the physical status while theobserver 910 observes the observed person 915, by comparing theplurality of images of the observed person 915 which are captured withina predetermined period of time. When the observer 910 observes aplurality of observed persons, the physical status quantifying section715 quantifies a change of the physical status of each of the observedpersons, and calculates the average value for the quantified changes.

For example, the image extracting section 710 extracts an image of theobserved person 915 which is captured when the observer's point of sightposition judging section 705 judges that the observer 910 looks at thearea corresponding to the eyes of the observed person 915, and an imageof the observed person 915 which is captured when a predetermined timeperiod has elapsed after the time at which the above image is captured.The physical status quantifying section 715 compares the imagesextracted by the image extracting section 710 in terms of the size ofthe pupil of the observed person 915, to calculate the change in thesize of the pupil. Subsequently, the physical status quantifying section715 quantifies the calculated change in the size of pupil. For example,the physical status quantifying section 715 may calculate, as the changein the size of the pupil, the rate of change from the size of the pupilof the observed person 915 which is detected at the time at which theobserver 910 looks at the area corresponding to the eyes of the observedperson 915 to the size of the pupil of the observed person 915 which isdetected when a predetermined time period has elapsed, with the formersize being used as a reference. Subsequently, the physical statusquantifying section 715 calculates an average value of the changescalculated for the plurality of observed persons. The physical statusquantifying section 715 may also quantify the change in the position ofthe point of sight of the observed person 915 and the change in thenumber of blinks.

Furthermore, for each of the diseases stored on the image storingsection 320 in association with the images of the observed person 915which are extracted by the image extracting section 710, the physicalstatus quantifying section 715 compares the image of the observed person915 which is stored on the image storing section 320 in association withthe disease, with the image of the observed person 915 which is storedon the image storing section 320 in association with informationindicating that the disease is not found. The physical statusquantifying section 715 may quantify the difference in the physicalstatus of the observed person 915 based on the result of the comparison,to calculate physical status difference quantified data. The physicalstatus quantifying section 715 may obtain the difference in the size ofthe pupil by comparing the physical statuses shown by the observedperson 915 when the observed person 915 suffers from a disease and doesnot, and quantify the obtained difference, to generate physical statusdifference quantified data.

The physical status quantifying section 715 may obtain the difference inthe size of the pupil of the observed person 915 by comparing the sizesof the pupil shown when the observed person 915 suffers from a diseaseand does not, and quantify the obtained difference. In addition, thephysical status quantifying section 715 may obtain a difference inchange of the position of the point of sight of the observed person 915between when the observed person 915 suffers from a disease and when theobserver person 915 does not suffer from the disease, and quantify theobtained difference. Also, the physical status quantifying section 715may obtain a difference in the number of blinks of the observed person915 between when the observed person 915 suffers from a disease and whenthe observed person 915 does not suffer from the disease, and quantifythe obtained difference. The physical status quantifying section 715quantifies the physical statuses of the plurality of observed persons,and calculates an average value based on the total of the quantifiedphysical statuses. The physical status quantifying section 715 may usethe calculated average value as physical status quantified data. Thephysical status quantifying section 715 supplies the physical statusquantified data and the physical status difference quantified data tothe physical status data storing section 720.

The physical status data storing section 720 stores thereon, for eachdisease, the physical status quantified data received from the physicalstatus quantifying section 715. The physical status data storing section720 supplies the physical status quantified data to the quantified dataextracting section 730, physical status comparing section 735, andlikely disease output control section 750. The physical statuscalculating section 725 analyzes the image of the person underobservation which is received from the image input section 602, tocalculate and quantify the physical status of the person underobservation. Specifically speaking, the physical status calculatingsection 725 quantifies the physical status of the person underobservation. The physical status calculating section 725 may also have afunction as a physical status difference calculating section forcalculating the difference in physical status of the person underobservation by analyzing the image of the person under observation whichis captured when the person does not suffer from a disease and receivedfrom the image input section 602 and a newly captured image of theperson under observation.

To be specific, the physical status calculating section 725 quantifiesthe physical status of the person under observation shown in the imageof the person under observation, by analyzing a plurality of images ofthe person under observation which are received from the image inputsection 602. In more detail, the physical status calculating section 725may quantify the physical status, by comparing the images of the personunder observation which are captured within a predetermined period oftime in the chronological order. For example, the physical statuscalculating section 725 may calculate the number of blinks of the personunder observation within a predetermined period of time, and use thecalculated number as the quantified physical status. The physical statuscalculating section 725 supplies the information indicating thequantified physical status of the person under observation to thephysical status comparing section 735.

The physical status comparing section 735 compares the informationindicating the physical status of the person under observation which isreceived from the physical status calculating section 725, with thephysical status quantified data stored on the physical status datastoring section 720. The physical status comparing section 735 maycalculate a coincidence level as the result of the comparison. To bespecific, the physical status comparing section 735 compares theinformation indicating the quantified physical status which is receivedfrom the physical status calculating section 725, with the physicalstatus quantified data stored on the physical status data storingsection 720. For example, the physical status comparing section 735 maycalculate a coincidence level, with an assumption that the maximumcoincidence level is achieved when the information indicating thequantified physical status which is received from the physical statuscalculating section 725 coincides with the physical status quantifieddata stored on the physical status data storing section 720. Thephysical status comparing section 735 may also has a function of aphysical status difference comparing section for comparing thedifference in physical status of the person under observation which iscalculated by the physical status calculating section 725 with thephysical status difference quantified data stored on the physical statusdata storing section 720. The physical status comparing section 735supplies the result of the comparison to the likelihood calculatingsection 740 and likely disease output control section 750.

The likelihood calculating section 740 calculates the likelihood of eachdisease based on the coincidence level indicated by the result of thecomparison done by the physical status comparing section 735. To bespecific, the likelihood calculating section 740 judges a disease forwhich the result of the comparison done by the physical status comparingsection 735 shows the highest coincidence level, to be the most likelydisease which the person under observation may suffer from. For example,the likelihood calculating section 740 calculates a likelihood of 100%or the like for the most likely disease. The likelihood calculatingsection 740 may judge a plurality of diseases for which the result ofthe comparison done by the physical status comparing section 735 shows ahigher coincidence level than a predetermined level, to be likelydiseases which the person under observation may suffer from. Thelikelihood calculating section 740 supplies the calculated likelihood tothe likely disease output control section 750. The likely disease outputcontrol section 750 causes the output section 50 to output informationindicating a disease stored on the physical status data storing section720 in association with the physical status quantified data which showsa higher coincidence level than a predetermined level according to theresult of the comparison done by the physical status comparing section735.

The quantified data extracting section 730 extracts physical statusquantified data which is stored on the physical status data storingsection 720 in association with information indicating the disease inputthrough the disease information input section 604. The quantified dataextracting section 730 supplies the extracted physical status quantifieddata to the disease image generating section 745. The disease imagegenerating section 745 generates an image of a patient suffering fromthe disease indicated by the information input through the diseaseinformation input section 604, based on the physical status quantifieddata received from the quantified data extracting section 730. To bespecific, the disease image generating section 745 may include therein adisease template image storing section storing thereon template imageseach uniquely defined for a patient suffering from a certain disease.The disease template image storing section stores thereon templateimages each showing a patient suffering from a certain disease. Thetemplate image stored on the disease template image storing section maybe a still or moving image showing a patient suffering from a certaindisease.

The disease image generating section 745 extracts a template image whichis stored on the disease template image storing section in associationwith the disease indicated by the information input through the diseaseinformation input section 604. Subsequently, the disease imagegenerating section 745 generates an image of a patient suffering fromthe disease, based on the physical status indicated by the physicalstatus quantified data received from the quantified data extractingsection 730 and the extracted template image. Here, the disease imagegenerating section 745 may generate a moving image showing the patientsuffering from the disease. For example, for a certain disease, thephysical status shown by a patient with the disease is a change inposition of the point of sight within a predetermined period of time. Inthis case, the disease image generating section 745 generates a movingimage showing the patient with the disease who moves the line of sightbased on the template image and the change in the position of the pointof sight which is indicated by physical status quantified data. Theimage output control section 755 causes the output section 50 to outputthe image generated by the disease image generating section 745.

From the perspective of privacy, it may not be allowed to record animage of the person 917 with a disease in a digital format. According tothe present embodiment, however, it is not necessary to record the imageof the patient suffering from a disease. Alternatively, the presentembodiment can generate the image of the patient suffering from adisease based on the physical status quantified data calculated by thephysical status calculating section 725 and a template image provided inadvance.

Under the control of the likely disease output control section 750, theoutput section 50 outputs information indicating a disease which isstored on the physical status data storing section 720 in associationwith physical status quantified data which shows a higher coincidencelevel than a predetermined level according to the result of thecomparison done by the physical status comparing section 735. Forexample, the output section 50 outputs the name of a disease which theperson under observation is most likely to suffer from, which isidentified as a result of the comparison done, by the physical statuscomparing section 735, between the change in the position of the pointof sight and physical status quantified data indicating a change in theposition of the point of sight. The output section 50 may outputinformation indicating a plurality of disease, together with therespective likelihoods of the plurality of diseases which are calculatedby the likelihood calculating section 740. To be specific, the outputsection 50 outputs the names, symptoms and other relating information ofthe diseases in descending order of likelihood. In addition, the outputsection 50 outputs the image generated by the disease image generatingsection 745, which is received from the image output control section755. The output section 50 may output information indicating a diseasewhich is stored on the physical status data storing section 720 inassociation with physical status difference quantified data that shows ahigher coincidence level than a predetermined level according to theresult of the comparison done by the physical status differencecomparing section.

The image recording apparatus 10 relating to the present embodimentquantifies the physical status of the observed person 915 when theobserver 910 observes the observed person 915, and records thequantified physical status. Therefore, the image recording apparatus 10can compare the physical status of a newly observed person underobservation with the quantified physical status that has been recordedthereon, to output information indicating a disease which the personunder observation is likely to suffer from. In this way, the imagerecording apparatus 10 can assist the observer 910 to identify anaccurate disease when the observer 910 observes the observed person 915.

Also, when input with a name of a disease, the image recording apparatus10 relating to the present embodiment can generate an image showing apatient suffering from the disease corresponding to the input name basedon the image of the observed person 915 or person under observation andthe physical status quantified data obtained by quantifying the physicalstatus of the observed person 915 or person under observation. In thisway, the user of the image recording apparatus 10 can learn how toobserve the observed person 915 in terms of diseases by comparing thegenerated image and the observed person 915 or person under observation.

The image recording apparatus 10 relating to the present embodimentincludes therein the observer's point of sight position judging section705. In other embodiments, however, the image recording apparatus 10 mayforgo the observer's point of sight position judging section 705, and beconfigured in such a manner that the observer's point of sight judgingsection 250 has a function of the observer's point of sight positionjudging section 705. If such is the case, the observer's point of sightjudging section 250 judges whether the observer looks at the observedperson when the image of the observed person is captured, and the imageextracting section extracts an image of the observed person for whichthe observer's point of sight judging section 250 makes positivejudgment.

FIG. 29 illustrates an exemplary function of the physical statuscalculating section 725 relating to the present embodiment. Here, thephysical status calculating section 725 may also has a function of aphysical status difference calculating section for calculating adifference in the physical status of a person under observation bycomparing an image of the person under observation which is capturedwhen the person does not suffer from a disease and received from theimage input section 602 and an image of the person under observationwhich is newly captured.

To begin with, the physical status calculating section 725 analyzes theimage of the person under observation which is received from the imageinput section 602. For example, the physical status calculating section725 analyzes the size of the pupil of the person under observation basedon images captured within a predetermined period of time, that is tosay, a time period from the time t0 to the time t30 as shown in a graph1000. For example, the physical status calculating section 725 analyzesand identifies the pupil diameters of pupils 1010, 1020, 1030 and 1040of the person under observation at the times t0, t10, t20 and t30. Thephysical status calculating section 725 calculates a change in pupildiameter for the times t10, t20 and t30 with respect to the pupildiameter at the time t0, and adds together the calculated changes.Subsequently, the physical status calculating section 725 may use theresult of the addition as the physical status of the person underobservation.

FIG. 30 illustrates an exemplary structure of data stored on thephysical status data storing section 720 relating to the presentembodiment. The physical status data storing section 720 stores thereona plurality of pieces of physical status quantified data in a one-to-onecorrespondence with a plurality of diseases. The physical statusquantified data may include pieces of data generated by quantifyingvalues such as a change in the size of the pupil of the observed person915, a change in the position of the point of sight, a change in thenumber of blinks within a predetermined period of time, and a change intwitching of the eyelids or face. For example, the physical status datastoring section 720 stores thereon a plurality of pieces of dataindicating the movement of the eyes of the observed person 915 inassociation with each disease. The plurality of pieces of data includesline of sight data indicating a change in the position of the point ofsight, pupil data indicating a change in the size of the pupil, andblink data indicating a change in the number of blinks. The physicalstatus data storing section 720 may store thereon, for each disease,physical status quantified data obtained by the physical statusquantifying section 715. For example, the physical status data storingsection 720 stores thereon, for a disease 780, pieces of physical statusquantified data respectively in the sections of line of sight data,pupil data, and blink data.

Referring to the pieces of physical status quantified data for thedisease 780, the values indicated decrease in the order of the line ofsight data, blink data and pupil data. This means that, when theobserved person 915 is observed in terms of the disease 780, theobservation can be conducted with a focus on the line of sight data. Byexclusively looking at the pieces of line of sight data stored on thephysical status data storing section 720, the values of the pieces ofphysical status quantified data decrease in the order of the diseases780, 782, and 784. Therefore, the physical status comparing section 735can judge which one of the diseases 780, 782 and 784 is most likely, bycomparing physical status quantified data obtained by quantifying achange in the position of the point of sight, with the pieces ofphysical status quantified data (i.e. the pieces of line of sight data)stored on the physical status data storing section 720. For example, thephysical status comparing section 735 calculates the likelihoods of thediseases by dividing the pieces of physical status quantified data (thepieces of line of sight data) stored on the physical status data storingsection 720 by the physical status quantified data (the line of sightdata) received from the physical status calculating section 725. Thephysical status comparing section 735 then supplies, to the likelydisease output control section 750, a name of the disease which showsthe highest likelihood according to the result of the calculation. Thelikely disease output control section 750 outputs the received name ofthe disease to the output section 50.

It should be noted that the image recording apparatus 10 relating to thepresent invention can be utilized not only for the observation in themedical field but also in the fields of beauty treatment and sports, forexample. When the image recording apparatus 10 is applied in the areasof beauty treatment and sports, the term “disease” is used in a broadersense than “illness” or “sickness”. For example, when the imagerecording apparatus 10 relating to the present invention is used in thearea of beauty treatment, a disease indicates an observation resultunfavorable for the user, such as wrinkles, rough skin, and the like.When the image recording apparatus 10 relating to the present inventionis used in the area of sports, a disease indicates incorrect form and abad habit in each type of sports.

When the image recording apparatus 10 relating to the present inventionis used in the beauty treatment field, the image storing section 320stores thereon images showing different areas of a human body which havechanged with advancing age. The physical status quantifying section 715calculates the characteristics of the diseases such as “wrinkles”,“pigmented spots”, and “rough skin” based on image processing, todetermine the types of diseases, for example. The physical statusquantifying section 715 also quantifies the degree of aging which isindicated by the diseases such as “wrinkles”, “pigmented spots”, and“rough skin”. The observation result input section 610 is used to input,as the observation result indicating a disease, a result of observingthe observed person in terms of beauty treatment, such as “the personhas wrinkles” or “the person has a rough skin”. The physical statuscalculating section 725 quantifies an input image of a person underobservation in the same manner as the physical status quantifyingsection 715. The physical status comparing section 735 compares thequantified data of the person under observation with the data stored onthe physical status data storing section 720, to determine the degree ofthe wrinkles or skin roughness of the person under observation.

The disease image generating section 745 may prestore thereon, as thetemplate images, the images showing different areas of the human bodywhich have changed with advancing age. For example, the disease imagegenerating section 745 prestores thereon “an image showing the outercorner of the eye of a female in her teens”, “an image showing the outercorner of the eye of a female in her twenties”, “an image showing theskin of a female in her teens”, “an image showing the skin of a femalein her forties” and the like. The physical status calculating section725 calculates the characteristics of diseases based on an input imageof a person under observation. Based on the result, the template imagesstored on the disease image generating section 745 may be changed. Inthis manner, the output section 50 may display diseases the person underobservation is likely to suffer from in the future. For example, theoutput section 50 displays an image showing wrinkles the person underobservation may possibly have in five years' time without beautytreatment. As described above, the image recording apparatus 10 can beused to identify symptoms of a patient, when used in the area of beautytreatment.

When the image recording apparatus 10 relating to the present inventionis used in the sports field, the image storing section 320 storesthereon moving images showing forms corresponding to differentadvancement levels of a certain sport. Here, the following descriptionis made using the hitting forms in baseball as an example. The physicalstatus quantifying section 715 calculates, based on image processing,the characteristics of incorrect forms, such as “hitting only with theforce of the hands”, “the hitter does not lower the chin” and “thehitter does not keep an eye on the ball”, so as to determine thefeatures of the forms. The physical status quantifying section 715 mayquantify the degree of the incorrectness of the forms. The observationresult input section 610 is used to input, as the observation resultindicating a disease, the result of observing the observed person suchas “hitting only with the force of the hands”, “the hitter does notlower the chin” and “the hitter does not keep an eye on the ball”. Thephysical status calculating section 725 quantifies an input moving imageof a person under observation in the same manner as the physical statusquantifying section 715. The physical status comparing section 735compares the quantified data of the person under observation with thedata stored on the physical status data storing section 720, todetermine the degree of the incorrectness of the form of the personunder observation. As described above, the image recording apparatus 10can be used to identify the habit of a player, when used in the area ofsports.

FIG. 31 illustrates an exemplary hardware configuration of the imagerecording apparatus 10 relating to the present invention. The imagerecording apparatus 10 is constituted by a CPU surrounding section, aninput/output (I/O) section and a legacy I/O section. The CPU surroundingsection includes a CPU 1505, a RAM 1520, a graphic controller 1575, anda display device 1580 which are connected to each other by means of ahost controller 1582. The I/O section includes a communication interface1530, a hard disk drive 1540, and a CD-ROM drive 1560 which areconnected to the host controller 1582 by means of an I/O controller1584. The legacy I/O section includes a ROM 1510, a flexible disk drive1550, and an I/O chip 1570 which are connected to the I/O controller1584.

The host controller 1582 connects the RAM 1520 with the CPU 1505 andgraphic controller 1575 which access the RAM 1520 at a high transferrate. The CPU 1505 operates in accordance with programs stored on theROM 1510 and RAM 1520, to control the constituents. The graphiccontroller 1575 obtains image data which is generated by the CPU 1505 orthe like on a frame buffer provided within the RAM 1520, and causes thedisplay device 1580 to display the obtained image data. Alternatively,the graphic controller 1575 may include therein a frame buffer forstoring thereon image data generated by the CPU 1505 or the like.

The I/O controller 1584 connects, to the host controller 1582, thecommunication interface 1530, hard disk drive 1540 and CD-ROM drive 1560which are I/O devices operating at a relatively high rate. Thecommunication interface 1530 communicates with a difference device via anetwork. The hard disk drive 1540 stores thereon programs and data to beused by the CPU 1505 provided in the image recording apparatus 10. TheCD-ROM drive 1560 reads programs and data from a CR-ROM 1595, andsupplies the read programs and data to the hard disk drive 1540 via theRAM 1520.

The I/O controller 1584 is also connected to the ROM 1510, flexible diskdrive 1550 and I/O chip 1570 which are I/O devices operating at arelatively low rate. The ROM 1510 stores thereon a boot program executedby the image recording apparatus 10 at the start up, programs unique tothe hardware of the image recording apparatus 10, and the like. Theflexible disk drive 1550 reads programs and data from a flexible disk1590, and supplies the read programs and data to the hard disk drive1540 via the RAM 1520. The I/O chip 1570 is used to connect a variety ofI/O devices such as the flexible disk drive 1550 via, for example, aparallel port, a serial port, a keyboard port, a mouse port or the like.

An image recording program to be supplied to the hard disk drive 1540via the RAM 1520 is provided by a user in a state of being stored on arecording medium such as the flexible disk 1590, CD-ROM 1595 and an ICcard. The image recording program is read from the recording medium,installed via the RAM 1520 in the hard disk drive 1540 in the imagerecording apparatus 10, and executed by the CPU 1505. The imagerecording program to be installed in and thus executed by the imagerecording apparatus 10 causes the CPU 1505 and the like to operate theimage recording apparatus 10 as the image-capturing unit 20, storingunit 30, sound unit 40, output section 50, input unit 60 and analyzingunit 70 and the constituents of these units and section 20 to 70described with reference to FIGS. 1 to 30.

While the embodiments of the present invention have been described, thetechnical scope of the invention is not limited to the above describedembodiments. It is apparent to persons skilled in the art that variousalternations and improvements can be added to the above-describedembodiment. It is also apparent from the scope of the claims that theembodiments added with such alternations or improvements can be includedin the technical scope of the invention.

1. An image output apparatus for assisting an observer who observes anobserved person, comprising: an image-capturing section that captures animage of the observed person; an observer's point of sight measuringsection that measures a point of sight of the observer when theimage-capturing section captures the image of the observed person; anoutput section that outputs the image of the observed person which iscaptured by the image-capturing section; and a point of sight outputcontrol section that causes the output section to output informationindicating the point of sight of the observer which is measured by theobserver's point of sight measuring section, together with the image ofthe observed person which is captured by the image-capturing section. 2.The image output apparatus as set forth in claim 1, wherein the point ofsight output control section causes the output section to output a markindicating the point of sight of the observer which is measured by theobserver's point of sight measuring section, in such a state that themark is overlapped onto the image of the observed person which iscaptured by the image-capturing section.
 3. The image output apparatusas set forth in claim 1, further comprising a point of sight imageextracting section that extracts a partial image including the point ofsight of the observer which is measured by the observer's point of sightmeasuring section, from the image of the observed person which iscaptured by the image-capturing section, wherein the point of sightoutput control section causes the output section to output an imageobtained by enlarging the partial image extracted by the point of sightimage extracting section, together with the image of the observed personwhich is captured by the image-capturing section.
 4. The image outputapparatus as set forth in claim 3, further comprising a view measuringsection that measures a view of the observer, wherein the point of sightimage extracting section extracts an image corresponding to the view ofthe observer which is measured by the view measuring section, from theimage of the observed person which is captured by the image-capturingsection.
 5. The image output apparatus as set forth in claim 1, furthercomprising: an observed person's information storing section that storesthereon the image of the observed person which is captured by theimage-capturing section in association with the observed person; and aprevious image output control section that, when the image-capturingsection captures a new image of the observed person, causes the outputsection to output a previous image of the observed person which isstored on the observed person's information storing section inassociation with the observed person, together with the new image of theobserved person which is captured by the image-capturing section.
 6. Theimage output apparatus as set forth in claim 5, wherein the observedperson's information storing section further stores thereon the point ofsight of the observer which is measured by the observer's point of sightmeasuring section in association with the observed person, and theprevious image output control section causes the output section tooutput the information indicating the point of sight of the observer,together with the previous image of the observed person.
 7. The imageoutput apparatus as set forth in claim 5, further comprising: a soundrecording section that records sound information of the observer whenthe image-capturing section captures the image of the observed person;an image storing section that stores thereon the sound information ofthe observer which is recorded by the sound recording section, inassociation with the image of the observed person which is captured bythe image-capturing section; a sound information comparing section thatcompares sound information of the observer which is newly recorded bythe sound recording section with the sound information of the observerwhich is previously recorded by the sound recording section and storedon the image storing section; and a previous image extracting sectionthat extracts, based on a result of the comparison done by the soundinformation comparing section, a previous image of the observed personwhich is stored on the image storing section in association withprevious sound information of the observer which shows a highercoincidence level than a predetermined level with the sound informationof the observer which is newly recorded by the sound recording section,wherein the previous image output control section causes the outputsection to output the previous image of the observed person which isextracted by the previous image extracting section, together with thenew image of the observed person which is captured by theimage-capturing section.
 8. The image output apparatus as set forth inclaim 1, wherein the output section is a display section, the imageoutput apparatus further comprises: an observed person's informationstoring section that stores the image of the observed person which iscaptured by the image-capturing section in association with the point ofsight of the observer which is measured by the observer's point of sightmeasuring section; a previous image output control section that readsthe image of the observed person which is stored on the observedperson's information storing section, and causes the display section todisplay the read image of the observed person; and a viewer's point ofsight measuring section that measures a point of sight of a viewer whoviews the image of the observed person which is displayed on the displaysection, wherein the observed person's information storing sectionfurther stores thereon information indicating the point of sight of theviewer which is measured by the viewer's point of sight measuringsection, in association with the image of the observed person which isread by the previous image output control section.
 9. The image outputapparatus as set forth in claim 8, further comprising: a viewer's pointof sight judging section that judges whether the point of sight of theobserver which is measured by the observer's point of sight measuringsection coincides with the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section; and a viewedimage extracting section that, when the viewer's point of sight judgingsection judges negatively, extracts an image corresponding toinformation indicating that the points of sight do not coincide witheach other, wherein the previous image output control section causes theoutput section to output the image which corresponds to the informationindicating that the points of sight do not coincide with each other andis thus extracted by the viewed image extracting section.
 10. The imageoutput apparatus as set forth in claim 9, wherein the viewer's point ofsight judging section judges that the point of sight of the observerwhich is measured by the observer's point of sight measuring sectiondoes not coincide with the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section, when thepoint of sight of the viewer is not present within a predeterminedregion including the point of sight of the observer.
 11. The imageoutput apparatus as set forth in claim 9, wherein when the point ofsight of the observer which is measured by the observer's point of sightmeasuring section moves to a plurality of different positions, theviewer's point of sight judging section judges that the point of sightof the observer coincides with the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section if the pointof sight of the viewer moves to vicinities which respectively correspondto the plurality of different positions.
 12. The image output apparatusas set forth in claim 11, wherein the viewer's point of sight judgingsection judges that the point of sight of the observer coincides withthe point of sight of the viewer if the point of sight of the viewerwhich is measured by the viewer's point of sight measuring section movesto the vicinities in a same order as the point of sight of the observer.13. The image output apparatus as set forth in claim 10, wherein theviewer's point of sight judging section judges that the point of sightof the observer coincides with the point of sight of the viewer if thepoint of sight of the observer which is measured by the observer's pointof sight measuring section and the point of sight of the viewer which ismeasured by the viewer's point of sight measuring section are bothpresent in a predetermined region of the image of the observed personwhich is captured by the image-capturing section.
 14. The image outputapparatus as set forth in claim 13, wherein the predetermined region ofthe image includes at least one of an eye, a mouth and a hand of theobserved person.
 15. The image output apparatus as set forth in claim 8,wherein the point of sight output control section causes the outputsection to output at least one of (i) a mark indicating a position ofthe point of sight of the observer which is measured by the observer'spoint of sight measuring section and (ii) a mark indicating a positionof the point of sight of the viewer which is measured by the viewer'spoint of sight measuring section, in such a state that the at least oneof the marks is overlapped onto the image of the observed person.
 16. Animage output method for assisting an observer who observes an observedperson, comprising: capturing an image of the observed person; measuringa point of sight of the observer during the image capturing; andoutputting information indicating the point of sight of the observerwhich is measured in the point of sight measuring, together with theimage of the observed person which is captured in the image capturing.17. A computer readable medium storing thereon an image output programfor an image output apparatus that assists an observer who observes anobserved person, the image output program causing the image outputapparatus to function as: an image-capturing section that captures animage of the observed person; an observer's point of sight measuringsection that measures a point of sight of the observer when theimage-capturing section captures the image of the observed person; anoutput section that outputs the image of the observed person which iscaptured by the image-capturing section; and a point of sight outputcontrol section that causes the output section to output informationindicating the point of sight of the observer which is measured by theobserver's point of sight measuring section, together with the image ofthe observed person which is captured by the image-capturing section.